Go GC Pauses in an RTB Bidder: Mark Assist, Deadlines, and the Rust Decision

A crucial aspect of RTB bidder performance is understanding the different contributing factors to auction timeouts. The exchange deadline, which encompasses the round trip time, is the target metric for bid responses. Bidder performance can be analyzed by separating the budget from the collector, the collector from the scheduler, and the rewrite from the component that deserves attention.

The round trip to the exchange and back involves various components, including internet latency, connection setup, time in the accept queue, and deserializing requests. The internal deadline set by the exchange is lower than the tmax value due to the overhead of the bidder. A key observation is that a single connection's performance can be affected, with garbage collection occurring concurrently and impacting every connection.

A collector serves every connection, making a single connection a distinct fault. Garbage collectors are process properties, causing cycles triggered on any connection to charge every connection. The key to addressing this issue lies in understanding the different types of pause charges in the Go runtime. Two primary sources to consider are stop-the-world pauses, which affect all goroutines at once, and mark assists, which impact the goroutine responsible for the allocation.

Mark assists are particularly relevant, as they occur when allocation is fast, and background marking gets a fixed quarter of the processors. The allocation rate, calculated by multiplying QPS by bytes per bid request, determines the extent of mark assists. Choosing the right Go repair strategy depends on whether the focus is on stop-the-world transitions, mark assists, or marking cost.

By addressing each contributing factor separately, it's possible to optimize bidder performance, ensuring bid responses arrive within the exchange's deadline.