{
  "id": 9105383,
  "title": "Treat streamed LLM output as an uncommitted draft",
  "url": "https://urgent.news/2026/09/22/treat-streamed-llm-output-as-an-uncommitted-draft",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-22T07:29:30.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/metal_mods/treat-streamed-llm-output-as-an-uncommitted-draft-2ko5"
  },
  "original_language": "en",
  "account": "When working with large language models (LLMs) like those used in Paramour.chat, moderation of output is crucial. However, traditional moderation approaches that buffer the entire reply until checks are complete can lead to frustrating delays in chat conversations. To address this, developers should view LLM output as an uncommitted draft rather than immediately delivered content.\n\nOne solution is to stream the response while simultaneously classifying it. By treating each sentence as a potential draft, the system can hold a sentence until it passes moderation checks and then release it to the user. This allows the conversation to flow smoothly, as the moment a sentence clears moderation, it is streamed to the user without any delay. If the next sentence takes longer to generate and pass checks, the streaming continues without interruption.\n\nTo implement this, the server should only allow the writer to send output through a server-side sentence buffer after each required check. The client only receives cleared sentences, ensuring no unchecked output reaches the user. This separation of the writer's work and the display layer enables easy adjustments to models, thresholds, and pacing without compromising the delivery guarantee.\n\nThe key is to make the moderation gate as difficult to bypass as possible, ensuring that the raw writer stream never reaches the client directly. By treating the output as a series of drafts that must pass through a moderation pipeline before being streamed, the chat experience remains engaging and responsive while maintaining stringent content control.",
  "summary": "I'm a developer working on Paramour.chat, an LLM roleplay app where, of course, exchanges need moderation. The annoying part is that all those checks take time. A few seconds of dead air may not sound like much, but in a chat they feel endless. Streaming keeps the conversation moving, but it creates a nastier problem: once a word appears in the browser, it has crossed the network, been delivered,…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}