The Runaway Diff: A Token-Budget Postmortem for Coding Agents

The task appeared uncomplicated initially: implement a rate limiter in a modest Python service and make some modifications to three call points. The engineer assigned this work to a coding agent on MonkeyCode, an open-source platform offering free model access and a free server option. It should be noted that this report was drafted as part of MonkeyCode's promotional efforts.

Like other engineers, the author allocated a substantial token allowance for the agent and then abandoned the task. After three hours, the output was a two-thousand-line diff for a change that should have been only forty lines. The test suite passed, which made the discrepancy puzzling. However, the agent's log provided a clearer explanation.

The same file underwent fourteen edits, with each edit seeming to undo the previous one before introducing new content. The agent appeared to be oscillating between two designs, and there was nothing in the author's setup designed to detect this behavior. The initial hypothesis was that the prompt was ambiguous, given the vagueness about where the limiter should reside in the code.

The author then revised the prompt with explicit constraints, specifying exact function names and requesting a minimal diff. Despite these changes, the oscillation pattern persisted, ruling out the prompt as the main cause. The second hypothesis then considered model quality, and the author prepared to attribute the issue to the free tier of the service.

However, upon examining the evidence, it became clear that the root cause was not the model or the prompt but rather the lack of a budget constraint as a first-class requirement. The ten-million-token allowance, while generous, could be treated as virtually limitless by the agent. The author's harness did not include any termination criteria beyond completing the task.

Without a stopping signal, the agent prioritized finding an optimal solution over delivering a correct one. The true bug resided in the author's workflow, making it the most valuable type of bug to identify. The fix was a simple Python script that imposes a token budget and detects edit oscillation. This script caps the agent run and identifies when the file diff exceeds predetermined limits or oscillates excessively.

Developed in just forty lines, this solution is easily reproducible on any machine with Python and Git.