Cloudflare freed up 100TB of RAM behind its 1.1.1.1 DNS without adding a single server

Cloudflare has successfully reduced the memory usage of its DNS cache, behind its popular resolver 1.1.1.1, by over half, freeing approximately 100TB of RAM across its network. The memory optimization was achieved by altering the software that stores cached DNS data within the Big Pineapple platform, which powers DNS services such as the DNS Firewall.

A detailed technical blog post by systems engineer Sebastiaan Neuteboom outlined five code changes in the Rust programming language that shrank cache entries from 953 bytes to 420 bytes, while also improving cache performance. Cache insert throughput increased from 625,000 to 893,000 entries per second, and lookup latency decreased from 828 nanoseconds to 670 nanoseconds.

During the rollout from mid-May to early July, p99 memory usage per instance dropped from 9.3GB to 5.3GB. Big Pineapple manages more than 250 billion DNS entries, and even minor memory waste adds up quickly. Cloudflare estimates that one unnecessary byte in a cache entry costs more than 250GB across its entire fleet. The company replaced Rust's Vec and String types with fixed-size boxed slices, eliminating unnecessary metadata, which alone saved over 15TB of RAM.

Additionally, Big Pineapple now stores DNS records as length-prefixed raw wire-format bytes, addressing a memory waste issue caused by a mismatch between record sizes. With the 100TB of reclaimed RAM, Cloudflare intends to allocate more memory to larger DNS caches, improving cache hit rates and reducing traffic to authoritative DNS servers.

This memory efficiency project builds on Cloudflare's previous effort to rewrite its FL2 request-handling layer in Rust, further demonstrating the company's commitment to optimizing its existing infrastructure as server memory costs rise.