117 Ghost Errors: Anatomy of a Flaky AI Agent
Between May 15 and July 2 of this year, the session transcripts of our content pipeline accumulated at least 117 copies of the same error. File does not exist . One error class, 117 occurrences, spread across seven weeks of overnight runs. When I finally sat down and traced it, I found no bug. Not one line of code was doing anything other than what it was written to do. An error that fires 117…
Between May 15 and July 2, the content pipeline produced at least 117 instances of the same error. The error, "File does not exist," occurred across seven weeks of overnight runs on a single line. No bugs were found in the code, as each retry absorbed the error, converting it into costs. This error class was a new phenomenon that most software engineers had never encountered.
The pipeline generates technical articles overnight, using 18 agents for each topic, receiving a brief from deterministically generated TypeScript and Python code. The error arises from a nondeterministic failure in the agent's path resolution, where identical code and input file produce different outcomes on different runs. This bug is insidious because it is not visible in production, as retries absorb the failures.
The only way to detect the bug is by counting occurrences across runs, which was done by an IT analyst, not a software engineer.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
