{
  "id": 17882098,
  "title": "How We Eliminated Cache Stampedes in Node.js & Bun with Single-Flight Coalescing",
  "url": "https://urgent.news/2026/11/03/how-we-eliminated-cache-stampedes-in-node-js-bun-with-single-flight",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-03T00:49:54.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/zisamglitch/how-we-eliminated-cache-stampedes-in-nodejs-bun-with-single-flight-coalescing-2h0i"
  },
  "original_language": "en",
  "account": "Engineers who have scaled high-traffic backends have experienced the nightmare of expensive endpoints that are cached with a short TTL. When a cache key expires during a surge of requests, hundreds of simultaneous requests trigger a database storm, overwhelming the connection pool and causing latency spikes. The traditional solutions - probabilistic early expiration and distributed mutex locks - either waste CPU cycles or introduce network latency and deadlock risks.\n\nThe zero-overhead alternative is Single-Flight Promise Coalescing. When a cache miss occurs, the system checks if an execution promise for the key is already running. If so, it returns the existing promise and waits for the result. If not, it creates a new promise, fetches the data, caches it, resolves all awaiting calls, and then removes the promise. This approach eliminates lock contention and connection pool exhaustion while maintaining zero overhead.\n\nThe benchmark tested this method against traditional caching under 1,000 concurrent requests, with a simulated database query latency of 25ms. The Single-Flight Promise Coalescing method reduced database queries from 1,000 to just one, achieving a 99.90% reduction in DB load and zero connection contention. KVDock, an open-source caching engine, implements this solution with multi-tier storage support and native polyglot implementations for various runtimes.",
  "summary": "Every engineer who has scaled a high-traffic backend has lived through this nightmare: You have an expensive endpoint (e.g., aggregating leaderboard data or rendering a personalized dashboard). To protect your database, you cache the result with a 60-second TTL. Everything runs smoothly until the key expires while receiving 1,500 requests per second . Within a 50-millisecond window, hundreds of…",
  "key_points": [
    "Single-Flight Promise Coalescing eliminates cache stampedes",
    "Reduces database queries from 1,000 to 1 under 1k concurrent requests",
    "KVDock open-source caching engine implements this solution"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}