{
  "id": 3537891,
  "title": "The Model's JSON Was Almost Valid. I Made It Grade Its Own Homework for 48 Hours.",
  "url": "https://urgent.news/2026/08/26/the-models-json-was-almost-valid-i-made-it-grade-its-own-homework-for",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-26T15:49:32.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/codepy_1473/the-models-json-was-almost-valid-i-made-it-grade-its-own-homework-for-48-hours-30bn"
  },
  "original_language": "en",
  "account": "The experiment aimed to test a language model's ability to self-correct JSON payloads that were almost valid. The JSON often came back nearly correct, but the errors were hidden until they triggered validation failures. The workflow involved feeding the model a prompt that asked it to return specific fields as JSON. After generating the JSON, a validator checked it against a predefined schema. If the validation failed, the model received the original payload, the incorrect JSON, and the error details for a second attempt. All actions, both successful and unsuccessful, were recorded in a JSON Lines log. The process ran for 48 hours on a free server using a free model, with both passes utilizing the free model access. The setup was designed to expose the bugs that often hide in almost valid JSON responses.",
  "summary": "Every extraction pipeline I have ever pointed at a language model shares the same dirty secret: the JSON comes back almost valid. Almost is where the bugs live, because almost passes your eyes and then fails your schema at midnight. So I built a loop where the model grades its own homework, then let it run for 48 hours on a free server to see what breaks. The experiment The idea was simple: take…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}