Realtime Event Retention Policy: Scaling Delivery Tracking Maps Through 3 Recovery Windows

Realtime event retention plays a crucial role in scaling delivery tracking maps. The policy involves three recovery windows: a short, lossless stream for reconnects, a compact snapshot for backfill, and discarding movement noise once the map has caught up. The key is to distinguish between transient hints and business transitions.

A courier's typing indicator only needs seconds of retention, while a read receipt requires a durable state transition. GPS markers demand the latest position, not every recorded point while the phone is underground. Treating all these as events leads to inefficiency and slow reconnects.

Retention tiers should be based on actual usage patterns like reconnect frequency and the oldest useful update in your fleet. Implement a recovery contract where every message includes a monotonically increasing sequence, event type, and server timestamp. Use per-delivery or per-channel counters to avoid unnecessary contention.

Implement a bounded stream with expiry based on server time. On reconnect, reject older cursors with a "snapshot_required" response and send the current delivery state followed by new events. Provide a snapshot containing necessary information such as marker position, status, last receipt state, and field versions. Use idempotent apply and log cursor, selected branch, and snapshot version but exclude sensitive data like driver notes or tokens.

During an outage scenario, test the policy by simulating a lost handset connection and verifying the client renders the latest snapshot, applies deltas in order, and shows the receipt state. Measure the performance of fan-out during backfill and test the policy with time by freezing the server clock and reconnecting with "oldest - 1".