Why is my LLM stream empty? A field guide to broken SSE responses

When utilizing OpenAI-compatible APIs with streaming enabled, it's not uncommon to encounter an empty stream that appears to complete successfully without any error or exception. In my investigation of dozens of such cases, I have identified four common failure modes that lead to this issue. Here is a comprehensive guide to troubleshooting these problems and restoring proper functionality.

1. Reasoning-only responses: Certain models generate their entire answer within a reasoning channel (the thought process) and mark the content channel as empty. The stream functions properly, but your parser and UI remain empty. The solution lies in inspecting all delta fields utilized by the model rather than just the content field. If your client only reads choices[0].delta.content, a reasoning-only response will be impossible to distinguish from an empty one.

2. Missing finish_reason: In instances where a proxy or router truncates the final chunk, the finish_reason disappears silently, and many client libraries unintentionally overlook this message instead of raising an error. To address this, treat a missing finish_reason as an alarming sign rather than a minor quirk. Log and alert on these occurrences, as a stream ending without stop, length, or tool_calls did not terminate naturally but was instead abruptly cut off.

3. Malformed SSE framing: Although the Simple Stream Event (SSE) format appears straightforward—lines of data ending with data: [DONE]—it is not without its complexities. Multi-byte UTF-8 characters might be split across chunk boundaries, some proxies modify or remove the data: prefix, and chunks can arrive after [DONE] or the stream might end without it.

These issues can lead to data loss or unexpected program behavior. The remedy is to log the raw frames prior to parsing. By examining the raw log, you will capture the actual truth, even if downstream parsers fail to do so.

4. Truncated tool calls: When constructing tool calls from multiple deltas, agents can suffer from partial calls if the stream abruptly ends. This leaves agents waiting indefinitely for arguments that will never arrive. The fix involves accumulating arguments across deltas, validating the assembled call, and setting a timeout for any hanging tool states.

Regardless of the specific failure mode, the underlying issue remains the same: the protocol layer deceives you by reporting success while delivering a broken payload. The key to resolving these problems lies in observing the raw stream first and then analyzing the protocol. To streamline this diagnostic process, I have developed an open-source toolkit called agent-stream-doctor, which records OpenAI-compatible streams and automatically identifies the failure class.

You can download and install agent-stream-doctor using pip, and it provides 16 test cases running in CI on Python 3.9–3.13. Moreover, the toolkit operates offline on captured stream files, allowing you to diagnose production failures that occurred hours or days earlier.

To install agent-stream-doctor, simply run:

pip install git+https://github.com/Mohammad-Hasan-Kaman/agent-stream-doctor.git

Once installed, you can record a live stream to a file using the following command:

agent-stream-doctor record --base-url https://api.example.com/v1/chat/completions --model your-model --message Hello!

Finally, analyze the captured stream with:

agent-stream-doctor analyze stream.txt

By following these steps and utilizing the provided toolkit, you can effectively troubleshoot and rectify broken SSE responses, ensuring a seamless experience when interacting with OpenAI-compatible APIs. For more detailed information about my work and potential remote opportunities, please visit my portfolio at mhkaman.com.