How We Eliminated Cache Stampedes in Node.js & Bun with Single-Flight Coalescing
Every engineer who has scaled a high-traffic backend has lived through this nightmare: You have an expensive endpoint (e.g., aggregating leaderboard data or rendering a personalized dashboard). To protect your database, you cache the result with a 60-second TTL. Everything runs smoothly until the key expires while receiving 1,500 requests per second . Within a 50-millisecond window, hundreds of…
Engineers who have scaled high-traffic backends have experienced the nightmare of expensive endpoints that are cached with a short TTL. When a cache key expires during a surge of requests, hundreds of simultaneous requests trigger a database storm, overwhelming the connection pool and causing latency spikes. The traditional solutions - probabilistic early expiration and distributed mutex locks - either waste CPU cycles or introduce network latency and deadlock risks.
The zero-overhead alternative is Single-Flight Promise Coalescing. When a cache miss occurs, the system checks if an execution promise for the key is already running. If so, it returns the existing promise and waits for the result. If not, it creates a new promise, fetches the data, caches it, resolves all awaiting calls, and then removes the promise. This approach eliminates lock contention and connection pool exhaustion while maintaining zero overhead.
The benchmark tested this method against traditional caching under 1,000 concurrent requests, with a simulated database query latency of 25ms. The Single-Flight Promise Coalescing method reduced database queries from 1,000 to just one, achieving a 99.90% reduction in DB load and zero connection contention. KVDock, an open-source caching engine, implements this solution with multi-tier storage support and native polyglot implementations for various runtimes.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
