{
  "id": 15273041,
  "title": "LLM JSON output: one setting dropped valid JSON from 75% to 4%",
  "url": "https://urgent.news/2026/10/17/llm-json-output-one-setting-dropped-valid-json-from-75-to-4",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-17T20:31:41.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/mattbusel/llm-json-output-one-setting-dropped-valid-json-from-75-to-4-1o12"
  },
  "original_language": "en",
  "account": "On October 17, 2026, a reporter conducted a test to determine the impact of various settings on language models' ability to produce valid JSON output. The test was conducted using three small language models - SmolLM2 360M, Qwen2.5 0.5B Instruct, and Llama 3.2 1B Instruct - on a Chrome browser in real-time using Transformers.js 4.2.0 with 4-bit weights. The models were asked to generate JSON 432 times, with the settings being varied and measured.\n\nThe reporter found that the no_repeat_ngram_size setting was the most detrimental to the models' ability to produce valid JSON. When this setting was enabled at size 3, the models' successful JSON output rates dropped to as low as 4%. The reporter concluded that to get reliable JSON output from an LLM, one should turn off the no_repeat_ngram_size setting and keep the repetition_penalty at 1.0 for any structured call, include the exact schema in the prompt, and parse with a forgiving ladder before validating against the schema.",
  "summary": "Originally published at tensorust.com . TL;DR: On October 17, 2026 I asked three small language models for JSON 432 times , in a real Chrome tab, with the exact sampling settings my browser chat app ships. Llama 3.2 1B returned schema-valid JSON 75% of the time with plain greedy decoding. Turning on one anti-repetition setting, no_repeat_ngram_size: 3 , dropped that to 4% . With all the chat…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}