How Netflix Scaled Its Real-Time Service Map

Netflix has detailed how it revamped its Service Topology real-time map to handle its production scale. The new design breaks the pipeline into three stages: intermediary resolution, enrichment, and persistence. Earlier, the system concentrated popular destinations, causing some instances to handle up to 100 times the usual traffic while also undergoing I/O-heavy enrichment. By separating these processes, the system can now redistribute the workload more efficiently.

To manage backpressure, Netflix employs Apache Pekko Streams, allowing pressure to travel upstream when graph storage can't keep up. This prevents dropping records or delivering incomplete maps, which would be preferable to stale data from batch-generated maps during incidents. Additionally, Netflix has replaced gRPC between pipeline stages with server-sent events (SSE).

SSE is lighter-weight and compatible with reactive backpressure, helping manage the high cost of serialization, connection-pool management, and streaming-response memory pressure at the company's volume. Each instance in the processing fleet reads from the same current list of healthy instances and uses consistent hashing to assign aggregators.

When an instance joins or leaves, the updated list reallocates only the affected aggregators, eliminating the need for a separate rebalancing process. Service Topology also uses time-windowed aggregator snapshots and property-level mutation history to reconstruct the topology at any specified point in time, aiding engineers in analyzing changes around incidents.