{
  "id": 9991816,
  "title": "Retry, Backoff, and Circuit Breakers for LLM API Calls",
  "url": "https://urgent.news/2026/09/26/retry-backoff-and-circuit-breakers-for-llm-api-calls",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-26T14:00:20.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/draganristicrsjpg/retry-backoff-and-circuit-breakers-for-llm-api-calls-h3k"
  },
  "original_language": "en",
  "account": "LLM APIs can fail in various ways, including rate limits (429), overload, timeouts, and degraded output. When building systems that interact with these APIs, it's essential to implement retry, backoff, and circuit breaker mechanisms to ensure resilience and prevent cascading failures.\n\nThe retry ladder suggests retrying on 429, 5xx, network errors, and timeouts for read-only calls, while carefully retrying on timeouts for calls with side effects, and never retrying on 400s, validation errors, authentication failures, or content policy refusals.\n\nExponential backoff with jitter and honoring the Retry-After header are crucial to avoid self-inflicted denial of service. Implementing a deadline cap on total retry time is also important. Circuit breakers should be used to handle a provider that is failing, by monitoring the failure rate and opening the circuit when it crosses a threshold.\n\nA persistent queue is essential to handle work arriving while the provider is down, ensuring that no important work is lost during an outage. The fallback model emphasizes uptime engineering, where degraded providers should be handled gracefully to prevent a single failure from impacting the entire system.",
  "summary": "> ../patterns/retry_backoff_circuit_breakers.md Retry LLM API calls on 429, 5xx, and timeouts with exponential backoff plus jitter, honor the Retry-After header when the provider sends one, cap total retry time with a deadline, and wrap the whole thing in a circuit breaker so a degraded provider cannot take your system down with it. Never retry non-idempotent actions blindly, and never retry…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}