Your Agent Loop Is Lying About Being Done: Writing Verifiable Stop Conditions
TL;DR — A loop is only as trustworthy as its stop condition. "Done" has to be checkable by a different model — tests pass, lint clean, schema validates — never the worker's own claim. Below: why self-grading fails, the four rungs of verification, a checker-agent template you can copy, and the three caps that stop a loop running forever. Part 3 of the Loop Engineering series on ShipWithAI. Read…
A loop's trustworthiness hinges on its stop condition. Self-grading can lead to premature declarations of completion, exemplifying the failure mode of Anthropic's agents. A proper stop condition must be verifiable, falsifiable, and cheap. The author introduces a four-rung verification system: bare while loops with exit codes, stop hooks, separate models grading each turn, and scheduled goal runs.
The key takeaway is separating the maker from the checker, which drastically improves verification reliability. The article provides a template for a checker-agent, emphasizing that the checker should have different instructions, optionally a different model, and ideally a clean context each turn. The article also presents a real-world example - a bug in the ShipWithAI's content-agent repo due to faulty keyword placement in the title.
The fix involves checking not just the target keyword but also the entire title, using a fresh-context maker turn and capturing the result with the bash test exit code. The loop is considered finished when a command exits with a 0 status, not when the agent feels finished.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
