{
  "id": 18581440,
  "title": "LLM Returns Invalid JSON Once in a While: Structured Outputs vs Validate-and-Retry",
  "url": "https://urgent.news/2026/11/06/llm-returns-invalid-json-once-in-a-while-structured-outputs-vs",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-06T22:31:25.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/libme/llm-returns-invalid-json-once-in-a-while-structured-outputs-vs-validate-and-retry-5faj"
  },
  "original_language": "en",
  "account": "When parsing JSON produced by an LLM occasionally fails, three main issues can be the root cause: truncation, extraneous text or code formatting around the JSON, and schema mismatches between the model output and expected structure. Structured outputs from providers help eliminate the latter two causes for certain schemas. To handle these failures, a validation-and-retry approach with a typed model is recommended, complemented by a logging mechanism for raw responses and stop reasons.\n\nThe most common error messages include JSONDecodeError indicating improper JSON structure, Unterminated string errors, and Pydantic validation errors pointing to specific issues like priority values exceeding limits. These errors have distinct fixes: removing code fences or brackets, adjusting schema limits, and enforcing strict type validation on the provider side.\n\nBefore modifying prompts, it's crucial to log raw output and stop reasons for each failure. Often, the cause is truncation, which cannot be resolved by prompt adjustments alone. The process begins by validating incoming data with a typed model. A function extracts the outermost JSON object, handling potential truncation or nested objects. Another function then attempts parsing using the model, catching validation errors and feeding them back to the model for correction.\n\nThe retry mechanism caps attempts to three to prevent excessive costs, and the assistant role provides the corrected JSON if validation fails. This approach ensures precise error feedback, minimizing manual intervention and optimizing LLM interactions in production environments.",
  "summary": "TL;DR: If json.loads() on model output fails only occasionally, there are three distinct causes: truncation, stray prose or code fences around the JSON, and schema drift (valid JSON, wrong shape). Provider-side structured outputs (constrained decoding) eliminate the second and third for supported schemas; a validate-and-retry loop with a typed model covers everything else, including the first. In…",
  "key_points": [
    "LLM JSON parsing failures can stem from truncation, extraneous text, or schema mismatches.",
    "Structured outputs and typed models help eliminate extraneous text and schema mismatches.",
    "Validation-and-retry approach with logging raw responses is recommended for handling failures."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}