Your Agent Says Verified. The Check Behind It May Be Empty.
Ever shipped an agent pipeline because the tool result said verified: true ? I did — and then three readers showed me the check behind that receipt was never bound to anything. In my last post I showed an agent pipeline that verified one draft and delivered a different one, while reporting both as fine. The fix was a hash binding: the verifier records the sha256 of the bytes it checked, and the…
A software engineer discovered an issue with agent pipelines that reported "verified: true" even when the check behind the receipt was empty. The engineer found that the verifier was only recording the hash of the bytes it checked, not actually running the predicate to verify the content. This allowed a verifier to pass without actually checking the content.
The engineer created a grid of tests using three different tools and frameworks, all with the same pipeline structure: building a draft, verifying it, and delivering it. The drafts were intentionally built from a known bad artifact, so they would violate the predicate. The verifier's claim of verification did not match the fact that the check was empty.
The engineer fixed the issue by naming the check on the receipt, authoring the check outside the delivery path, and feeding a known bad artifact through the same check inside the same run. This exposed the empty verifier and confirmed that honest limitations of the check could be exposed by using a known bad artifact.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
