The Cache Stampede Problem: Why a Popular Cache Key Can Take Down Your Backend
Caches are supposed to protect your backend from load. Most of the time they do. But there is a specific failure mode where the cache itself becomes the trigger for an outage: the cache stampede, also known as the thundering herd problem. It shows up in systems that otherwise look healthy, which is what makes it so disruptive when it finally happens. What a Cache Stampede Is A cache stampede…
When a frequently accessed cache entry expires or gets evicted, multiple requests to fetch the updated value can all hit the backend simultaneously, causing a cache stampede or thundering herd problem. This occurs when a cache miss is treated uniformly by every request, leading to repeated recomputation of the same data point. Common scenarios include hot keys with a hard time-to-live (TTL), a cache that's empty following a deployment or restart, or a popular key becoming viral due to sudden traffic spikes.
For example, an API endpoint that generates a leaderboard from a 800-millisecond database query might see 500 requests per second all hitting the database at once when its cached value expires. To mitigate this issue, engineers often employ various strategies. They may use locks to serialize recomputation, or implement "single flight" techniques to ensure only one request performs the work while others wait.
Adding random variations to TTLs can help spread expiration times, and serving stale data while refreshing it in the background can provide faster responses without overwhelming the backend. More advanced methods like probabilistic early expiration trigger background refreshes just before expiry, minimizing the chance many requests are served from the cache at the exact moment it becomes stale.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
