Why a 99% Cache Hit Rate Is 10x Faster Than 90% (Not 9%)
I ran the same query 10,000 times: almost 2 seconds of database work for an answer that never changed. The database did not fail. I just kept asking it the same question. Before fixing it, I wrote down a prediction: a cache should make this about 100 times faster. Keep that number. It is wrong in a way that teaches something. The whole pattern is a map Cache-aside is the oldest trick in the book:…
A 99% cache hit rate is significantly faster than a 90% hit rate. This was demonstrated by running a query 10,000 times, with almost 2 seconds of database work for an unchanging answer. The prediction of a 100x speed increase due to caching was incorrect; the actual increase was 3,084x faster. This is because the database only needed to execute the query once, while the cache reduced the need for real reads.
The cache made the query rare instead of faster. The cache hit rate, or the share of reads the cache answers, is crucial in determining performance. In the experiment, hit rates of 0%, 50%, 90%, and 99% were simulated, showing that the time measured went from 1,993.9 ms to 20.1 ms as the hit rate increased. While it seems like a 9% improvement from 90% to 99%, the time reduction was actually 10x due to the cache reducing the number of misses.
However, cache staleness can be an issue, as cached data may become outdated. A time-to-live (TTL) can be set to limit the cache's answer age, but even with a TTL, some data may become stale, making it harder to balance staleness and performance.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
