Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
Article URL: https://blog.cloudflare.com/dns-cache-memory-optimization-1111/ Comments URL: https://news.ycombinator.com/item?id=49468083 Points: 277 # Comments: 79
Big Pineapple, the platform behind 1.1.1.1 DNS service, stores over 250 billion DNS cache entries across its fleet. By optimizing the cache's memory footprint, they were able to free up roughly 100 terabytes of memory. Five changes to the cache storage process reduced the per-entry footprint by over 50%, resulting in significant memory savings.
These changes also improved cache performance, with throughput increasing by 43% and lookup latency dropping 19%. The optimizations proved especially impactful for DNS queries using EDNS Client Subnet (ECS). Instead of storing multiple versions of the same query, the service now stores just one with offsets for the response sections.
This change saved 28 bytes per cache entry. Overall, the optimizations led to substantial memory and performance improvements, equivalent to 15 terabytes of memory and faster lookup speeds.
Written by urgent.news from Hacker News Best's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
This story
This is one outlet's version. Read the fullest account.
- How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache blog.cloudflare.com
