Retry budgets by language: Python, Go, and JavaScript
Originally published on Loop & Retry — field notes on building LLM agents that survive production. The retry-budget argument is language-independent: your retry multiplier is set by how you recover, not by how often you fail, and a per-call cap bounds a call but never a run or a fleet of workers . That's the theory, grounded in the $200 cost of nested caps that multiplied across layers . The…
The strategy for setting a retry budget hinges on shared state, not the frequency of failures. The budget is a token bucket that refills slowly with successful calls, and withdrawals happen during retries. If the bucket is empty, a retry is denied. This design ensures retries are a privilege, not a right, and caps them based on successful outcomes.
In Python, a decorator is often used to encapsulate the retry logic. However, this can lead to a per-call cap, as each function call gets its own independent four-attempt budget. This fails to achieve the desired shared budget across all retry attempts. A better approach is to create a RetryBudget class that maintains shared state, with methods to refilling tokens based on success or failure and allowing retries only if tokens are available.
Failure to use a lock when accessing shared state can lead to race conditions under concurrent execution, undermining the budget's effectiveness.
In Go, the retry logic is more explicit, with the retry context passed alongside a shared *RetryBudget. The budget is part of the context, allowing the context's deadline to serve as the budget's expiry, while the budget object manages its own retry count. This design makes it clear that the budget is shared and governed by the context's deadline. If the budget's token count hits zero, retries are halted, respecting the budget's limits irrespective of the number of concurrent retry attempts.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
