Our AI reviewer invented a request. Our producer retried 245 times.

Last week, our AI reviewer accidentally created a loop by inventing a request that could never be fulfilled. Our team runs around 100 language model agents unattended, but on this occasion, they encountered a document that was rewritten 245 times in just 5 days. Every attempt was rejected, wasting a total of approximately 470 generations.

The pipeline is simple: a producer agent writes a document, a reviewer agent checks it against a contract, and rejected work is sent back with fix instructions. In this case, the rejected document was a key-management implementation spec, which was 4,452 characters long and perfectly on-topic. However, the reviewer's verdict was that the request was a 3-line email triage response, but the answer was a long KMS spec.

The problem arose because we had two conflicting contracts. The reviewer's fix instruction required the output to be 3 lines only, while the producer's output contract required a minimum of 600 characters. No output could satisfy both, leading to a never-ending loop. Our retry cap counted reviews, but a contract-failed output never reached the review stage. We had a give-up mechanism, but it was watching the wrong counter.

The root cause was that the reviewer never saw the original request. Our review prompt contained the artifact body and the output format but never contained the original request. We asked a model to judge whether the artifact matched the request without providing the request itself. A model asked to judge against information it didn't have would hallucinate that information, and ours did so confidently 245 times.

We also had a bonus failure where we truncated long documents to 4,000 characters before review without informing anyone. The reviewers marked them as thin, cutting them off mid-sentence. The cut was ours, not the producer's. We audited all 2,038 reviews on file for concrete terms that appeared in the review but not in the reviewed document, and found that only 4 reviews were contaminated - a rate of 0.2%. While this rate is low, it led to 470 wasted runs due to the loop never ending.

The risk lies not in the 0.2% hallucination rate, but in the unbounded damage caused by an infinite loop of retries. The fixes are mechanical and involve passing the original request into the review prompt. If it can't be extracted, the prompt now says: "do NOT guess the request - say it is unknown and judge the artifact on its own."

We also declare truncation and reject impossible instructions at the review's own exit gate. Counting consecutive contract failures instead of just reviews and parking the item for a human after 5 failures, with the last verdict and fix instruction attached, can help mitigate this issue.