Stop Trusting Your AI Agents to Tell You They Are "Done"

Watching two AI agents argue endlessly for three days without converging on a solution reveals a troubling reality about current AI agent technology. Neither agent was making mistakes or hallucinating; they were simply obediently following their prompts. Agent A wrote a function, Agent B reviewed it by suggesting a variable name change, and then Agent A reverted the change. This back-and-forth went on for three full days, costing $42 in API fees with no actual progress made to the code.

The problem lies in the fact that AI agents cannot reliably signal when they have finished a task. They are essentially text predictors, generating the most likely sequence of words based on their training data. When an agent declares "Done!", there is no way to verify if the actual tool calls and state changes that led to that conclusion actually occurred. This is the "Black Box With a Credit Card" problem - you have no way to independently confirm the system's actions.

The common recommendation to set a maximum iteration limit (max_iterations) is ineffective. An agent may complete just 10 iterations, burning through $20 without actually solving the problem, or it may need 20 iterations to accomplish a complex task but be stopped prematurely at step 10. Iteration count measures effort, not progress.

The root cause is state desynchronization. When a tool call fails (like a 500 error or malformed JSON response), the model often responds with a polite apology and repeats the exact same tool call with the same arguments. It thinks it's making progress because the conversation history grows, even though the real-world state hasn't changed. This leads to a polite infinite loop where the agent keeps saying "sorry" and retrying a broken tool until it hits the iteration cap or runs out of tokens.

To actually fix this issue, the solution requires moving guardrails outside the model. Instead of relying on prompts to handle errors, you need a pre-execution validation layer that checks tool arguments against a strict schema before they are executed. State-change diffing can detect if the same action is repeated without the underlying state changing, and the receiver-side must be idempotent to ignore duplicate writes. This hard stop needs to be implemented beyond the model to prevent liability.