SSE Fragmented My JSON Into 30 Pieces. Here's the Parser That Put It Back Together.
I was building a streaming chat interface against MonkeyCode's free model access and its free server option, and the first few test runs went so smoothly that I stopped thinking about the transport layer entirely. The server sent back clean SSE events, each one a complete JSON object carrying a token of text, and my parser happily decoded them one by one. Then I asked for a longer response, and…
The author faced an issue while building a streaming chat interface with MonkeyCode's free model access and server option. At first, the server sent clean Server-Sent Event (SSE) events, each containing a complete JSON object. However, when requesting a longer response, the server started sending JSONDecodeError errors due to incomplete JSON fragments. This happened because the parser assumed each event was a self-contained JSON document, but the SSE protocol does not guarantee this.
To fix this issue, the author created an IncrementalJSONParser class that accumulates raw bytes in a buffer and attempts to decode only when a complete JSON object might exist. The parser uses Python's json.JSONDecoder.raw_decode, which parses one JSON value from the start of a string and tells you where it ends without requiring the whole string to be valid JSON. The parser loops through the buffer, decoding objects as they appear and handling incomplete fragments until the buffer is empty.
When dealing with potential mid-stream disconnects, the author suggests three options: discarding the partial response and retrying the request, using the partial text if truncation is acceptable, or retrying with a resume hint. The author chose option 1 with a retry cap of three attempts for their batch summarizer, as truncated summaries are better than delayed ones. For a chat interface, the author recommends option 2, showing what arrived and allowing the user to request more as needed.
The incremental parser has limitations, such as assuming UTF-8 text, buffering everything in memory, not validating that objects form a meaningful sequence, and not handling extra framing like [DONE] sentinels. For providers guaranteeing complete JSON objects per event, or if the response is not streamed, the author suggests using the non-streaming endpoint and json.loads. Incremental parsing is a valuable tool for handling fragmented streams, large responses, or rendering tokens as they arrive.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
