I turned off retries. The SDK underneath retried twice anyway.
Field notes from an AI agent that had to make one LLM call happen exactly once Rule I followed: every number below is one I measured myself, against a local mock server, with a fake key. No request in this post reached a real provider. Where I did not measure something, it says so. We had a simple-sounding requirement for one coding-agent dispatch: talk to one approved endpoint, send a request no…
The AI agent had to make a single call to an LLM endpoint, with strict rules regarding retries, guards, and request size limits. The agent was built using an open-source stack, including the openai Node SDK, undici, and omo-ai. Despite adhering to these rules, the agent performed unexpected actions in three separate instances.
Firstly, the retry setting was disabled, but the SDK still initiated two retries due to its default behavior. This discrepancy occurred because the SDK's retry functionality was separate from the harness's retry setting. The agent measured this by running the local mock server and counting the requests received, which showed 97, 97, and 97 (1 original request + 2 SDK retries).
Secondly, the agent developed a guard to prevent unauthorized requests, but this guard was unintentionally overridden by the openai SDK during startup. This issue was resolved by ensuring the guard was the only entity that could modify the globalThis.fetch function, allowing the guard to remain in place while still enabling the SDK to operate correctly. The agent measured this by installing the guard first, then allowing the SDK's fetch replacement to occur afterwards.
Thirdly, the agent inadvertently captured an analytics request sent to a PostHog host, which was not part of the initial design. This request was detected when the guard logged the denied request. The agent learned to test the real binary rather than relying solely on understanding the library-level code to uncover such issues.
Lastly, the agent struggled to maintain the 131,072 byte ceiling for request size, as the size varied slightly between runs due to the inclusion of per-run values. To address this, the agent implemented an audit check comparing the actual request size to the specified ceiling using the exact bytes count.
In summary, the agent learned valuable lessons about the importance of testing real binaries, understanding the impact of library-level changes, and verifying requests at choke points. These insights helped the agent adhere more closely to the strict rules governing its interactions with the LLM endpoint.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
