How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache

Big Pineapple, the platform behind 1.1.1.1 DNS services, stores over 250 billion DNS cache entries at any given time. By optimizing how cache entries are stored in memory, they managed to cut the per-entry footprint by over 50%, freeing up roughly 100 terabytes of memory across their fleet. This optimization also improved cache throughput by 43% and reduced lookup latency by 19%.

The cache operates by starting empty and filling as DNS queries arrive, evicting older or less popular items when it reaches maximum capacity. When EDNS Client Subnet (ECS) is used, multiple versions of the same query are cached, increasing memory usage. Each cache entry is a key-value pair, with the key identifying the queried entity and the value storing the DNS response along with metadata.

To improve efficiency, several changes were made. Vec and String fields, which carry overhead not needed once stored, were replaced with Box[T] and Box[str], saving 8 bytes per field and 64 bytes per entry. This reduced wasted heap space and eliminated excess capacity. Additionally, fixed-size offsets replaced separate lists for DNS record sections, eliminating redundant pointers and lengths. The owner field, often identical to the queried domain, was removed when possible, further reducing memory usage.