Building an x402-aware crawler in Go: what Cloudflare's Monetization Gateway means for web scraping

Cloudflare has reclaimed approximately 100TB of RAM without altering any physical RAM modules in its servers. The company achieved this by redesigning the memory layout of each DNS cache entry. Systems engineer Sebastiaan Neuteboom detailed five Rust-level changes in a technical blog post, which reduced the size of each cached entry from 953 bytes to 420 bytes, while simultaneously increasing the cache's performance.

Insert throughput improved by 43%, and lookup latency decreased by 19%. These changes equate to the combined DDR5 memory of 130 servers, making the most of the high cost of server-grade DDR5 memory. The first of the five changes involved replacing Rust's growable Vec and String containers with fixed-size boxed slices, which saved over 15TB of memory by eliminating capacity fields that were never used once cached.

Other modifications included collapsing three record lists in every DNS response into one buffer, indexing them with 2-byte offsets, and dropping redundant owner names that duplicated the queried domain. This last change, storing record data as length-prefixed raw wire-format bytes, also improved CPU cache locality, resulting in faster copy operations from the stored buffer to outgoing responses.

Cloudflare's benchmarks revealed that insert throughput increased from 625,000 to 893,000 entries per second, while lookup latency decreased from 828 to 670 nanoseconds. Over the rollout, which spanned from mid-May to early July, p99 resident memory per instance fell from 9.3GB to 5.3GB. This memory-efficient strategy is Cloudflare's second large memory-reclamation project in the past year, following the FL2 rewrite of its request-handling layer in Rust last September.

The company continues to prioritize efficiency in its hardware choices, such as switching to AMD's 96-core EPYC 9684X for its Gen 12 servers. The reclaimed RAM will be used to bolster larger DNS caches, increasing hit rates and reducing query traffic to authoritative servers.