{
  "id": 2905508,
  "title": "Our AI reviewer invented a request. Our producer retried 245 times.",
  "url": "https://urgent.news/2026/08/24/our-ai-reviewer-invented-a-request-our-producer-retried-245-times",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-24T00:31:03.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/gxcafellc/our-ai-reviewer-invented-a-request-our-producer-retried-245-times-465f"
  },
  "original_language": "en",
  "account": "Last week, our AI reviewer accidentally created a loop by inventing a request that could never be fulfilled. Our team runs around 100 language model agents unattended, but on this occasion, they encountered a document that was rewritten 245 times in just 5 days. Every attempt was rejected, wasting a total of approximately 470 generations.\n\nThe pipeline is simple: a producer agent writes a document, a reviewer agent checks it against a contract, and rejected work is sent back with fix instructions. In this case, the rejected document was a key-management implementation spec, which was 4,452 characters long and perfectly on-topic. However, the reviewer's verdict was that the request was a 3-line email triage response, but the answer was a long KMS spec.\n\nThe problem arose because we had two conflicting contracts. The reviewer's fix instruction required the output to be 3 lines only, while the producer's output contract required a minimum of 600 characters. No output could satisfy both, leading to a never-ending loop. Our retry cap counted reviews, but a contract-failed output never reached the review stage. We had a give-up mechanism, but it was watching the wrong counter.\n\nThe root cause was that the reviewer never saw the original request. Our review prompt contained the artifact body and the output format but never contained the original request. We asked a model to judge whether the artifact matched the request without providing the request itself. A model asked to judge against information it didn't have would hallucinate that information, and ours did so confidently 245 times.\n\nWe also had a bonus failure where we truncated long documents to 4,000 characters before review without informing anyone. The reviewers marked them as thin, cutting them off mid-sentence. The cut was ours, not the producer's. We audited all 2,038 reviews on file for concrete terms that appeared in the review but not in the reviewed document, and found that only 4 reviews were contaminated - a rate of 0.2%. While this rate is low, it led to 470 wasted runs due to the loop never ending.\n\nThe risk lies not in the 0.2% hallucination rate, but in the unbounded damage caused by an infinite loop of retries. The fixes are mechanical and involve passing the original request into the review prompt. If it can't be extracted, the prompt now says: \"do NOT guess the request - say it is unknown and judge the artifact on its own.\" We also declare truncation and reject impossible instructions at the review's own exit gate. Counting consecutive contract failures instead of just reviews and parking the item for a human after 5 failures, with the last verdict and fix instruction attached, can help mitigate this issue.",
  "summary": "We run ~100 LLM agents unattended on local models. Last week we found one document that had been rewritten 245 times in 5 days — every attempt rejected. A sibling document: 225 times. Combined, about 470 wasted generations, all burned on the same two files. Here is the autopsy, with the actual numbers. The loop Our pipeline is simple: a producer agent writes a document, a reviewer agent checks it…",
  "key_points": [
    "AI reviewer created loop by inventing unfulfillable request",
    "Producer retried 245 times, wasting 470 generations",
    "Issue resolved by passing original request into review prompt"
  ],
  "editors_take": "The incident highlights a vulnerability in AI review systems where conflicting contracts and incomplete review prompts can create infinite loops, allowing a small hallucination rate to cause significant damage.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}