03 - Streaming LLM Tokens in PHP with Server-Sent Events

The worst part of a chat user interface is the blank screen while the model is generating its response. A solution to this problem is to stream the individual tokens as they are generated, rather than waiting for the entire reply to be produced. In PHP, this can be achieved using Server-Sent Events (SSE). NanoAgent's Agent::stream() method allows you to receive each token through a callback function. Your responsibility is to simply format the token as SSE and flush it.

To set up a streaming endpoint, you only need to handle the request when the URL contains "?stream=1&message=..." and include the necessary headers in the response. These headers inform the browser to maintain the connection and read the data incrementally. The essential headers are Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive, and X-Accel-Buffering: no (critical under Nginx).

Next, you need to initialize the NanoAgent with the appropriate configuration, including the model provider, model, and API key. Create a new instance of the Agent class and call the stream() method, passing the user's message and a callback function. Inside the callback, format the token as an SSE line and force PHP to send it immediately using echo, ob_flush(), and flush().

Once the model has finished generating its response, send a sentinel value (e.g., "[DONE]") to signal the client. Then, close out the request. It is crucial to include the ob_flush() and flush() calls after every token, as PHP buffers output by default. Without these calls, the browser won't receive anything until the request ends, which negates the purpose of streaming.

The client-side implementation is straightforward: create an EventSource instance targeting the streaming.php endpoint with the appropriate query parameters. Attach a message event listener that checks if the received data equals "[DONE]". If not, parse the token from the data, and append it to the answer element. This approach allows you to display the model's response in real-time, without the need for a WebSocket connection.