{
  "id": 13295705,
  "title": "Move Slow LLM Calls Off the Request Path with BullMQ in Node.js",
  "url": "https://urgent.news/2026/10/10/move-slow-llm-calls-off-the-request-path-with-bullmq-in-node-js",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-10T02:01:29.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/geminate_solutions_9b6035/move-slow-llm-calls-off-the-request-path-with-bullmq-in-nodejs-54fh"
  },
  "original_language": "en",
  "account": "Long prompt completions from large language models (LLMs) can take 20 seconds or more to complete. Embedding these lengthy tasks directly into an HTTP handler can cause numerous issues in a real-world application. Timeouts stack up across load balancers, reverse proxies, and clients. Retries may lead to duplicate processing and exceed provider rate limits. Restarting the API process could drop ongoing jobs. To mitigate these problems, move the slow LLM call to a separate queue with BullMQ, returning an immediate response while a dedicated worker handles the asynchronous processing. This approach allows a better retry policy, concurrency control, and failure handling without impacting the main request path.",
  "summary": "If an LLM call can take 20 seconds or more, keep it out of your HTTP handler. Put the work on a BullMQ queue backed by Redis, return a job ID right away with a 202 Accepted , and let a separate worker make the slow call while the client gets progress updates over Server-Sent Events. The rest of this post is working code for that setup. It covers retries with backoff, concurrency and rate limits,…",
  "key_points": [
    "Move slow LLM calls to a separate queue using BullMQ in Node.js",
    "Return immediate response while dedicated worker handles asynchronous processing",
    "Improves retry policy, concurrency control, and failure handling"
  ],
  "editors_take": "Moving slow LLM calls off the request path enables better handling of lengthy tasks, allowing for improved retry policies, concurrency control, and failure handling without impacting the main request path.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}