The Best Evidence for a Fact-Checking Process Is the Errors It Catches

The most confident sentence in an article draft I wrote this year was incorrect. It claimed that vendor SDK imports in an open-source project were limited to exactly four files. This claim was specific, verifiable, and survived my initial review—a common type of error found in published works. However, before the article was released, a separate fact-checking pass caught it, revealing the actual number was five files plus one type-only file.

This article served as a reminder that the effectiveness of a fact-checking process is best demonstrated by the errors it catches.

The pipeline behind my DEV series, Engineering WorldScript Studio, consists of three gates and a ledger. The first gate, Gate A, involves a package rather than a draft. Each article is compiled as a dossier containing a brief, outline, evidence file, and a claim ledger. Every factual claim gets a dedicated row in the ledger, listing the claim, its source file or API, the commit SHA it was verified against, and its status.

If a claim cannot find a row, it does not receive a sentence. The fact-check report then proceeds claim by claim, comparing the ledger against a fresh copy of the repository, not relying on memory or the draft's own assertions.

Gate B is the staging phase with verification. The draft is published without the ability to view it publicly, and the stored body is compared byte-for-byte with the source in the local repository. This practice was established following a minor incident where I pushed a small post-publish fix using a cached article body, which temporarily published two live articles.

The incident prompted a rule: always fetch the body fresh from the server. This practice is documented as D-29 in the ledger and will remain valid regardless of the tools used.

Gate C is the publish-day drift guard, which is the part of the process I find most valuable. Before the publishing process begins, the repository's HEAD is re-resolved, and every evidence file is byte-diffed against the verified baseline. A particularly notable near-miss occurred when I realized that the author had inadvertently missed a sixth file during inventory checks.

This oversight was found only after a re-grep of the entire repository on the day of publication. The rule now requires that no inventory claim can survive the scrutiny of a path-filtered grep.

Two instances illustrate the strengths of this process. First, the distinct drafting and fact-checking sessions ensure that the writer and claim-checker operate with different perspectives and assumptions. This separation caught the initial near-miss by re-running the enumeration process and discovering an additional file. Second, the publish-day drift guard and routine API re-check caught two subsequent near-misses.

These gates serve as learning experiences, as the near-misses caught by the first gate eventually become internal habits, remaining in place even after the external checks are no longer necessary.

Additional rules complement this pipeline but are less visible in the diagram. These include releasing only verified truths, blocking articles that describe unreleased code, maintaining a retired-claims register that documents every claim that has been disproven, and enforcing absolute claims with verification. By ensuring that all claims are backed by verifiable evidence, this fact-checking process demonstrates the rigor of the editorial process.