{
  "id": 20801763,
  "title": "Celery visibility_timeout: 214 LLM Jobs Ran Twice on Redis",
  "url": "https://urgent.news/2026/11/19/celery-visibility-timeout-214-llm-jobs-ran-twice-on-redis",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-19T20:52:49.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ji_ai/celery-visibilitytimeout-214-llm-jobs-ran-twice-on-redis-4f53"
  },
  "original_language": "en",
  "account": "Celery visibility_timeout determines how long the Redis transport waits for an acknowledgement before assuming a worker has failed and reinserting the message into the queue. Due to this timeout, Redis has no way of knowing whether a worker has simply run out of time on a long-running job or has crashed. Over 14 nights, Celery's default visibility_timeout of 3600 seconds caused 214 out of 1380 jobs to be run twice unnecessarily, as Redis repeatedly delivered the same message to different workers.",
  "summary": "The bug showed up as a spreadsheet problem. My nightly pipeline summarizes long PDFs with Claude and writes one row per document. One morning, document 4471 had two summaries. Slightly different wording, start times 61 minutes apart, same Celery task ID. Same task ID is the part that made my stomach drop. That isn't a retry. That's the broker handing the exact same message to a second worker…",
  "key_points": [
    "Celery visibilitytimeout set to 214 seconds",
    "Redis transport reinserts message into queue",
    "214 jobs run twice due to timeout"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}