Three layers of automated fact-checking for an LLM newsroom (and the bugs that forced each one)

Three layers of automated fact-checking were implemented for an LLM newsroom to avoid issues with AI-generated content. The first layer, an article-level critic, checks every draft against the source events it was built from. This involves identifying errors such as wrong attribution, merged or split entities, date errors, contradictions between dek and body, and claims based on a single source.

Any high severity finding triggers a single revision pass, aimed at correcting only the flagged error and preventing the introduction of new errors. This is because, as witnessed, unbounded self-revision leads to wandering rather than convergence.

The second layer, search-verified claim checking, comes into play when the critic fails to catch errors in the source data. This layer extracts every crucial claim – who, what, when, why, and number – and conducts a fresh news search for each claim. The search results are then reviewed by a neutral search query to determine if the claim is supported, wrong, contested, or unverified.

A crucial rule in this layer is that the model's world knowledge may flag a claim, but only retrieved sources can authorize a rewrite. If the model is confident about a claim's inaccuracy but the sources are silent, the claim will be hedged as contested or unverified, never rewritten from memory. This ensures that fabricated corrections are avoided.

The third layer focuses on auditing the data rather than just the output. It involves a rolling audit that re-reads each timeline event's original source article and checks for recurring failure classes. These include speculation stated as fact, contested claims stated flatly, later confirmation dates used as action dates, over-claims like "first known instance", and re-reports of actions already in the database with an earlier date.

Any such issues are rewritten, duplicates are merged via redirects, and mismatched sources are flagged. The last issue, where news wires re-report the same government action for weeks with different headlines and dates, was particularly challenging for the judge to detect.

The engineering behind these layers reveals some key lessons. The first is the importance of caping concurrency when dealing with claim searches, to avoid rate limiting and silent failures. The second is to measure overlap against the smaller of two event sets when preventing duplication, rather than the candidate's own list, as seen in the model's initial defeat of prompt instructions against duplication.

These insights highlight the necessity of enforcing verification rules in code, rather than relying solely on prompts, which may not always control the denominator accurately.