{
  "id": 2592331,
  "title": "Make Retry Idempotency an Invariant Before You Move a Batch Job to a Free Model Endpoint",
  "url": "https://urgent.news/2026/08/22/make-retry-idempotency-an-invariant-before-you-move-a-batch-job-to-a",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-22T14:05:50.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/robinzzz/make-retry-idempotency-an-invariant-before-you-move-a-batch-job-to-a-free-model-endpoint-ad0"
  },
  "original_language": "en",
  "account": "The article discusses a bug that occurred when moving a nightly summarization batch job to a free model endpoint, resulting in three emails, two database rows, and a confused customer. The log indicated a timeout after 30,000ms, a duplicate key in the database, and the three seemingly unrelated incidents were actually the same bug. The issue arose when the job was moved to the free endpoint, which had a wider tail in latency distribution due to shared capacity and cold starts. The retry loop generated a new idempotency key on each attempt, causing one logical operation to be executed three times physically. The invariant that was broken was idempotency - the consumer must apply the same logical operation exactly once, regardless of how many times the message is delivered. The article emphasizes the importance of measuring the distribution of latency instead of relying on the average, as timeouts should be based on the P99 of the endpoint, not the P50.",
  "summary": "Job 4817 was submitted exactly once, and it produced three emails, two database rows, and one very confused customer. The log said timeout after 30000ms , the database said duplicate key , and for three hours those looked like two unrelated incidents. They were the same bug, and it appeared the night we moved a nightly summarization batch to a free model endpoint to push the inference bill to…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}