Human Review Records for AI Decisions

Human Review Records for AI Decisions: What to Keep and How to Make Them Verifiable

A human review record is a stored note of what a person looked at, what they decided, and when, in relation to a specific AI output. If an AI system helps decide who gets a loan, an interview, or a benefit, the record shows that a person actually weighed the output and was not just clicking through. To be useful later, it needs to say who reviewed what, what the reviewer saw, what they decided, and when. It also needs to be protected against quiet edits.

That last part gets less attention than it should. Plenty of organisations log reviews. Far fewer can show, two years later, that the log is the same one that existed on the day of the decision

What a human review record should contain

The record should let a stranger reconstruct the review without asking anyone. In practice that means these items:

* A reference to the AI output being reviewed, such as a case ID, model version, and the output itself or a stable pointer to it.

* The inputs the reviewer was shown. If the interface showed a score and three highlighted factors, the record should say so.

* The reviewer's identity or role, according to your internal access rules.

* The decision: approved, overridden, escalated, or sent back.

* The reasoning, in the reviewer's own words, at least for overrides and borderline cases.

* Time of review and time the final decision was issued.

* Any time-on-task or interaction data you already collect, if you use it as a signal of real review.

Notice what's missing from that list: a lone "Reviewed: yes" field. That tells an auditor almost nothing. It's the review equivalent of a signature on a blank page.

Why these records matter

The regulatory reasons

Several EU rules point toward this kind of documentation, though none of them use the phrase "human review record."

The AI Act requires high-risk systems to be designed so that natural persons can effectively oversee them while in use (Article 14). Deployers must assign oversight to people with the competence, training and authority to do the job (Article 26). High-risk systems must also support automatic event logging (Article 12), and deployers must keep the logs under their control for a period appropriate to the system's purpose, with a floor of at least six months unless other law says otherwise (Article 26).

The timeline has changed. The original date for most high-risk obligations was 2 August 2026. The Digital Omnibus on AI, which entered into force in July 2026, moved the application date for stand-alone high-risk systems under Annex III (employment, education, credit and similar areas) to 2 December 2027, and for high-risk AI embedded in regulated products to 2 August 2028. Treat that as a planning horizon, not a reason to wait. Records of oversight can't be created retroactively, and a system that goes live in 2027 with no review documentation has a problem from day one.

GDPR is already in force and points the same way. Article 22 limits decisions based solely on automated processing that have legal or similarly significant effects, and the safeguards include a right to obtain human intervention. The Court of Justice's SCHUFA ruling (Case C-634/21) suggested that producing a score can itself count as the decision when a third party relies heavily on it. That raises the stakes on whether the human step is real. A rubber-stamped review may not take you outside Article 22.

This is general information, not legal advice. Which rules apply depends on your role, your sector, and your jurisdiction.

The practical reasons

Disputes, complaints, audits and internal investigations all ask the same question: what happened at the moment of the decision? A review record answers it. It also protects reviewers. If a person overrode a flawed recommendation, you want that on file in a form nobody can later claim was written afterwards.

The weak point: records that can be edited

Most review logs live in a database or case management tool. Administrators can change rows. Backups get restored. Fields get "corrected." None of that implies bad faith, but it means that when someone asks for proof the record is unchanged, the honest answer is often "trust our system."

You can improve on that by fingerprinting the record and anchoring the fingerprint somewhere you don't control.

How a timestamped review record works

The idea is simple. You export the finished review record as a file. A cryptographic hash function (SHA-256 is the common choice, specified in NIST's FIPS 180-4) turns that file into a fixed-length fingerprint. Change a single character in the file and the fingerprint changes completely. You then anchor that fingerprint to a public blockchain, which gives it a time reference that third parties can check without relying on your own systems.

Later, anyone holding the original file can recompute the hash and compare it with the anchored one. A match shows the file is bit-for-bit what existed at the recorded time. A mismatch shows something changed.

A hypothetical example?

Imagine a loan team whose reviewers close each case by exporting a PDF containing the model output, the reviewer's notes and the final decision. Case 4471 is closed at 14:32. The PDF is hashed and anchored. Eight months later a customer disputes the outcome. The team produces the PDF, an auditor recomputes the hash, and the result matches the anchored record. The team has shown the document existed in that exact form at or before the recorded time. (This is an invented scenario for illustration, not a real case.)

How Certelo handles it?

With Certelo, the SHA-256 hash is calculated locally on the user's device, so the review record itself is not uploaded just to create the proof. Only the hash is anchored to the Electra Protocol blockchain. For review records, which often contain personal data, that matters. The sensitive file stays inside your own environment, and what goes to the blockchain is a fingerprint. You can verify later using the original file, the hash, or the certificate details.

One practical tip applies to any hashing workflow. Export records in a fixed, repeatable format. If your system regenerates the PDF each time and embeds a creation timestamp, the bytes will differ and the hash will not match. Store the exact file you hashed.

Also think about the personal-data side. A hash of a record containing names and decisions is a one-way fingerprint, but a very short or predictable record could in theory be guessed and tested. Including a unique case ID and free-text reasoning in each record makes this a non-issue in practice. Your DPO should still sign off on the design.

What a timestamp does not prove

This is where many articles on the subject overreach, so it's worth being blunt.

A blockchain timestamp shows that a specific file fingerprint existed at or before a given time. It does not show that:

* the reviewer was competent, attentive, or even who the record says they were,

* the AI output was correct,

* the review was meaningful and not a formality,

* the decision was lawful,

* the record was complete, or that nothing relevant was left out before hashing.

It also doesn't make the record legally admissible by itself. Admissibility and evidential weight depend on the jurisdiction and the forum. And an anchored fingerprint is not an eIDAS qualified electronic timestamp, which is a separate, regulated concept. If your use case calls for one, a timestamp of that kind needs to come from a qualified provider.

What the anchoring does give you is narrow and useful: protection against the claim that the record was written or altered after the fact. Everything else still depends on good process.

Building a review process that produces credible records

A few habits make the difference between a record that survives scrutiny and one that doesn't.

Capture the reviewer's view, not just the verdict. If the reviewer saw a risk score and nothing else, say so. Reviews are only as meaningful as the information the reviewer had.

Record overrides in detail. Disagreement with the model is the strongest evidence that a human is actually in the loop. Track override rates too. A rate near zero deserves a question about automation bias.

Seal at a defined moment. Decide when a record is final, for example when the decision is issued, and hash it then. Later notes become new records, not edits.

Keep the chain of custody simple. Write down where the original file lives, who can read it, and how long you retain it, keeping the Article 26 minimum in mind where it applies.

Test verification before you need it. Pick a sealed record, recompute the hash, and confirm the match. Doing this once under no pressure is worth a lot.

Frequently asked questions

Does the AI Act require "human review records" by name?

No. The Act requires effective human oversight, logging, and deployer retention of logs for high-risk systems. Records of how oversight actually worked are the natural way to evidence that, but the exact format is up to you unless sector rules say otherwise.

Is a "reviewed by" checkbox enough?

Rarely. It shows a box was ticked, not that anyone evaluated the output. Records that capture what the reviewer saw, what they decided and why are far more convincing.

How long should review records be kept?

For deployers of high-risk systems the AI Act sets a floor of at least six months for logs under their control, unless other law provides otherwise. Sector rules, limitation periods and your own dispute exposure often call for longer. Check with counsel.

Can I timestamp records that contain personal data?

You can hash and anchor a file without publishing its content, since only the fingerprint is recorded. Whether that is acceptable for your data is a question for your data protection officer. Keep the original records in your own secured systems.

What happens if the file changes after it was hashed?

The recomputed hash will no longer match the anchored one, which is exactly the point. Any edit, even a single character, breaks the match.

Does this prove the reviewer did a good job?

No. It proves the record existed in a given form by a given time. Quality of review comes from training, workload, interface design and oversight of the process.

Summary

A good human review record says what the reviewer saw, what they decided and why. Anchoring its hash to a public blockchain adds something ordinary logs can't: a way for an outsider to check that the record hasn't been quietly rewritten. It won't prove the review was good or the decision lawful, but it closes off one common line of attack. If you want to try it on a sealed record, you can create and verify a timestamp with Certelo without uploading the file itself.