A green diff is not a completeness check
Every agent-authored change arrives with the same reassuring signal: the tests pass, the diff is small, nothing looks obviously wrong. And every so often one of those changes has quietly removed a column, a branch, or a configuration key that no test ever touched. The diff was honest. It reported exactly what it checked — the tests that exist. The gap is what a diff can never report: the…
A single change in software doesn't always guarantee the absence of issues. The change may pass all tests, yet still have unnoticed impacts, such as the removal of a column, branch, or configuration key. The diff, which reports only the tests that exist, cannot capture the completeness of a change relative to the surface it touches.
This is because deletion can be consistent with all test assertions: removing something that was never selected will not trigger any failures. Conversely, addition often breaks things quickly, as readers who are unaware of the new column start returning incorrect data or throwing errors. Reviewers usually look for new additions, mistaking them for potential mistakes in the code.
However, agents commonly make the opposite mistake: deleting what they believe to be unused code, which is often right about the tests but wrong about the actual usage. To address this issue, one approach is to create a separate diff that shows the changes in the surface, such as columns added, removed, or type-changed for databases, keys removed or defaulted differently for config files, and fields dropped from the response shape for APIs.
By reviewing this separate diff independently of the code change, reviewers can better assess the completeness of the change. To implement this, snapshot the surface before and after the change, and fail the check on any removal unless the same change set also touches every reader of that field. If readers cannot be enumerated mechanically, treat it as a reason to require human acknowledgment instead of skipping the check.
This approach converts the question "Did the agent drop something?" from a review question into a mechanical one that can be reliably addressed even during busy periods. Importantly, this is not an argument against deletion or against relying on diffs. Instead, it highlights that a diff provides information about what changed, while the review question is about what remains. These are two distinct claims, and only the former should be checked.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.