Coalescing the Stream

Claude Code does not communicate directly with a model provider. Instead, it utilizes the claude-code-router, a self-hosted proxy that allows the same CLI to point at either the official Anthropic API or third-party providers. The reporter observes the entire process through the VS Code extension over Remote-SSH. The previous article, Two Clocks, Neither Lying, explained why the previous path allowed an approval prompt to sit for an hour due to one streaming event per token, an extension struggling to process them quickly, and the CLI queuing control messages behind six thousand deltas.

This piece focuses on the solution, which involved two sequential steps: writing a middleware that merges streaming events back into chunks, and ensuring the middleware actually runs. The first step took an evening, while the second required five attempts. The middleware must merge consecutive content_block_delta events without altering the stream's meaning.

A successful merge only occurs when events share the same block index and delta type, and the window is flushed when any other event appears. The middleware does not understand the conversation; it only combines events that can be proved interchangeable. The window controls merging aggressiveness, with a global setting and per-type overrides to cater to different event types.

Keep-alive pings are dropped by default to prevent flushing the window at critical points in the stream. The middleware must merge only what it fully understands, declining the rest to avoid corruption. Two key transport truths must be maintained: the content-length header changes after merging, and compression must not be applied to merged bodies.

The original issue was a slow reader backing up a pipe, and a naive merger would just relocate the problem. The solution was to create a middleware that merges events without changing the stream's semantics, ensuring the downstream consumer only receives coherent data.