Tamper-Evident Logs for Security Investigations

Tamper-Evident Logs for Security Investigations: How to Make Log Evidence Hold Up

A tamper-evident log is one where any later change, deletion or reordering of entries can be detected, even though it can't be physically prevented. It usually works by linking each entry to the one before it with a cryptographic hash, so altering any record breaks every record after it. The detection is only trustworthy if the latest hash is also recorded somewhere the attacker can't reach, which is why external anchoring matters as much as the chain itself.

That last condition is where most implementations quietly fall short.

The problem investigators actually face

Picture the first hour of an incident review. You pull authentication logs from a server that was compromised three weeks ago. The entries look clean. No gaps, nothing odd. Do you trust them?

An experienced responder doesn't, at least not yet. Clearing or editing logs is a standard step after intrusion, and MITRE ATTACK catalogs it as Indicator Removal (technique T1070). The crude version leaves traces: Windows writes event 1102 when the Security log is cleared and 104 for other logs. The careful version is quieter. An attacker with root or administrator rights can delete selected lines, edit timestamps, or rewrite the file so it looks continuous.

So the question for an investigation isn't only "what do the logs say?" It's "can anyone show these logs are the same ones that existed before the incident?" Tamper-evident logging exists to answer that second question.

Tamper-proof is the wrong promise

You'll see "tamper-proof logs" in a lot of vendor copy. It's a promise nobody can keep for logs on a machine the attacker controls. If someone holds root, they can write to the disk, and no software layer on that same machine can fully stop them.

The more honest goal is tamper evidence. You accept that modification might happen, and you design so that it can't happen silently. Security researchers framed it this way in the late 1990s: Bruce Schneier and John Kelsey's work on secure audit logs aimed to keep an attacker who takes over a machine from undetectably rewriting what happened before the compromise. Crosby and Wallach later refined the data structures, using Merkle trees to make verification efficient at scale.

This distinction changes how you build things. Prevention controls (permissions, WORM storage, append-only filesystems) raise the cost of tampering. Evidence controls tell you whether it happened. A serious setup uses both, but only the second lets you say something defensible in a report.

How a hash chain turns a log into a chain of evidence

The mechanism is simpler than it sounds. Each log entry gets hashed together with the hash of the entry before it. The hash of entry 500 therefore depends on everything from entry 1 to entry 500.

Change one character in entry 200 and its hash changes. Entry 201 was built from the old hash of 200, so it no longer matches. So does 202, and every entry after. Verification is just recomputing the chain and comparing.

You can see the idea in a few lines of shell:

That's a teaching sketch, not production code. Real systems add sequence numbers, handle log rotation, and sign or protect the chain. But it shows why the technique is attractive: it's cheap, and it uses nothing more exotic than SHA-256.

For large volumes, Merkle trees do the same job more efficiently. Certificate Transparency (RFC 6962) is the best-known real-world deployment: public logs of certificates built on Merkle trees, where anyone can check that an entry is included and that the log has only grown. Its value comes from the fact that many independent parties watch it.

The gap hash chains leave open

Here is the weakness. A hash chain proves internal consistency. It doesn't prove the chain is the original one.

An attacker with write access can recompute the entire chain after editing it. The result is a perfectly valid chain containing falsified history. If the only copy of the final hash lives on the compromised system, you've detected nothing.

The fix is to push the latest hash somewhere the attacker can't alter. Typical options:

* Forward entries to a remote collector or SIEM as they're written, so the attacker has to compromise a second system.

* Write the head hash to a separate, access-controlled store or WORM bucket.

* Use a trusted timestamp authority under RFC 3161, which signs a hash together with a time.

* Anchor the hash to a public blockchain, so any third party can later confirm that the hash existed by that time.

Each has different trust assumptions. A remote collector relies on your own organization's security. An RFC 3161 authority relies on the authority's integrity and keys. A public blockchain anchor relies on nobody in particular, which is useful when your own organization's neutrality may be questioned, for example in a dispute with a vendor, a regulator or a former employee.

Where blockchain anchoring fits, and where it doesn't

Anchoring is a small, specific tool. You take the head hash of a sealed log segment, or the SHA-256 of the archived file itself, and record it on a blockchain. From then on, anyone with the file can recompute the hash and check it against the on-chain record. If it matches, that file existed in exactly that form at or before the block's time. If one byte differs, it doesn't match.

For investigations, that gives you something concrete: a time boundary outside the compromised environment. If the incident began on March 10 and your March 9 log archive was anchored on March 9, an attacker can't rewrite that archive afterwards without the mismatch showing.

It does not give you more than that. Anchoring says nothing about whether the log content was accurate when written. If a compromised agent logged lies in real time, the lies are faithfully preserved. It also doesn't prevent deletion. If the only copy of the archive is destroyed, you can no longer prove its contents, only that a hash was once recorded. Keep copies.

Using Certelo for log archives

Certelo's approach suits this particular job, with one caveat about scope. Certelo hashes a file locally with SHA-256, so the original file stays on the user's device and isn't uploaded just to create the proof. Only the hash is anchored to the Electra Protocol blockchain. For logs, that matters. Security logs routinely contain usernames, IP addresses, internal hostnames and sometimes tokens that slip into error messages. Being able to create a timestamp without shipping that data to a third party removes a real objection from legal and privacy teams.

A practical workflow, in plain terms:

1- Rotate or seal logs on a schedule (daily is common) and compress each period into one archive.

2- Optionally generate the hash-chain head for that period, and store it alongside the archive.

3- Run the archive through Certelo to create the blockchain timestamp record, and keep the certificate information with the archive.

4- During an investigation, hash the archive again and verify it against the record.

Two honest notes. The hash chain and the sealing process in steps 1 and 2 are things you build on your side; Certelo supplies the timestamp and verification layer in step 3. And a Certelo record establishes that this exact file existed by the recorded time. It doesn't establish who generated the logs, that the system clock was accurate, or that the logs are complete.

What a timestamp can and can't do in front of an auditor or a court?

Be careful here. Whether a particular record is accepted as evidence depends on the jurisdiction, the rules of the forum, and how the surrounding process was documented. A blockchain record is one piece of an evidentiary picture, not a verdict.

In EU law, for instance, eIDAS gives special legal effect to qualified electronic time stamps, which are issued by qualified trust service providers. A blockchain anchor isn't automatically that. That doesn't make it worthless; it can still support a record of integrity, but you shouldn't describe it as a qualified timestamp. If your obligations require one, check with counsel.

For handling digital evidence generally, ISO/IEC 27037 and NIST SP 800-86 describe identification, collection and preservation practices. Tamper-evident logging complements those practices. It doesn't replace chain-of-custody documentation.

A hypothetical example

The following is illustrative, not a real case.

A small SaaS company anchors a hash of its production auth logs every night. In April, a customer claims an admin account was used to export their data on a date the company says it wasn't. Months later, the dispute escalates. The company produces the archived log for that night, along with the Certelo record created the following morning. An independent reviewer hashes the archive, compares it to the on-chain record, and confirms the match.

What has been shown: the archive hasn't changed since the anchor was made. What hasn't: whether the logging pipeline captured every event, or whether the admin credentials were stolen. Those questions still need conventional investigation. But the argument "you edited the logs after the fact" is much harder to sustain.

Things that quietly break log evidence

Most failures aren't cryptographic. They're operational:

1- Unsynchronized clocks. If hosts disagree on time, the sequence of events across systems is unreliable no matter how well the hashes work. Use NTP and log clock drift.

2- Gaps before the anchor. Anything logged but not yet anchored is vulnerable. Shorter anchoring intervals shrink the window.

3- Logs that never leave the host. Local-only logs are the easiest to rewrite. Forward them off-box immediately.

4- No verification practice. An anchor nobody ever checks is decoration. Test the verification path before an incident, not during one.

5- Missing key and process documentation. An auditor will ask who could write to the logs and the archive. Be ready to answer.

NIST SP 800-92 remains the standard reference for log management practices in general, and the OWASP Logging Cheat Sheet is useful for application-level logging.

You may ask these

What's the difference between tamper-evident and immutable logs?

Immutable storage, such as WORM media or object-lock buckets, tries to stop modification. Tamper-evident logging detects modification after the fact. They complement each other, and immutable storage is also only as strong as the account permissions around it.

Can an attacker with root access defeat a hash chain?

They can rewrite the chain if no copy of the head hash exists elsewhere. That's why the latest hash needs to be sent off-host or anchored externally as soon as possible.

Does blockchain anchoring make logs admissible in court?

Not automatically. Admissibility depends on jurisdiction and process. An anchored hash can support a record that the file existed unchanged at a point in time, but it's evidence to be weighed, not a guarantee.

Does Certelo upload my log files?

No. The SHA-256 hash is calculated locally, and the original file isn't uploaded merely to create the record.

How often should logs be anchored?

It depends on your risk and volume. Daily is a common baseline. Higher-risk systems may anchor hourly, since only entries before the last anchor are protected.

What if I need to verify a log years later?

You need the original archive, since the hash must be recomputed from it. Store archives redundantly, and keep the certificate information with them.