Before You Buy Another Server: Squeezing a Read-Heavy API for Max RPS
Learn how to squeeze maximum RPS from a read-heavy API before buying more servers—using zero-serialization, Redis pipelining, and response streaming.
When an API slows down, many developers instinctively add more instances. However, this can hide inefficiencies and lead to excessive capacity. The goal was to optimize a real-time inventory API that could request hundreds of SKU-location records per request. Before adding instances, the first step was to identify how much work one instance could handle.
The experiments involved various optimizations:
1. Refraining from JSON serialization: Under load, a significant portion of CPU was used for object mapping, even though the API only needed to fetch and concatenate data bytes. By avoiding JSON parsing, CPU usage decreased, and the garbage collector had less work to do. Ingestion pipelines pre-rendered payloads at write time, reducing the need for per-request CPU.
2. Minimizing Redis round trips: Fewer, larger calls were more efficient than many small ones. Storing SKU-per-location values in a single Redis hash reduced the number of lookups. Using a single multiplexed connection instead of a connection pool simplified operations and reduced contention.
3. Stream responses as they arrive: By using a BlockingQueue to collect Redis results and writing them to the socket as they're received, the system avoided waiting for the entire response. This approach set no Content-Length, relying on chunked transfer encoding to allow early response delivery to the client.
4. Evaluating containers: The streaming design required more threads per request. Three containers were benchmarked: Tomcat, Jetty, and Netty. While Netty offered the highest capacity, the team chose Tomcat due to its simplicity and ease of operation. Despite the added complexity, the team could manage it within resources.
5. Choosing the load balancer: HTTP/2 offered benefits like multiplexed streams and header compression, but only if clients negotiated it. The load balancer was configured for TCP passthrough, taking advantage of long-lived connections and keep-alives without unnecessary HTTP parsing.
By addressing these areas, the API's performance was significantly improved, demonstrating that optimizing the low-hanging fruit can lead to substantial gains before pursuing more advanced optimizations.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
