{
  "id": 13661203,
  "title": "Making LLM ratings auditable: quotes, limitations, and \"can't judge\" instead of zero",
  "url": "https://urgent.news/2026/10/11/making-llm-ratings-auditable-quotes-limitations-and-cant-judge",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-11T06:28:28.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ashish_624379625242c5d263/making-llm-ratings-auditable-quotes-limitations-and-cant-judge-instead-of-zero-36be"
  },
  "original_language": "en",
  "account": "In a recent dev.to post, an author proposed a set of guidelines for LLM-generated ratings. The main points are to fix the evidence window, display the actual quotes, acknowledge limitations, and offer a \"can't judge\" button instead of generating a score. The goal is to stop highly confident-sounding information from being presented as factual evidence. This concept of auditability is particularly relevant in the field of vision rehabilitation for amblyopia, where the industry has often operated as a \"black box.\" Patients, especially adults, were exposed to opaque treatments like patches, red-blue glasses, and rigid clinic schedules. There was no dashboard, no control over content, and no visibility into what the dominant eye was seeing compared to the lazy eye. This is where the Amblyotube app, developed by Seven Sports for Meta Quest, comes in. By leveraging the headset's independent eyepieces, Amblyotube implements dichoptic vision training, turning the experience into a transparent layer over the user's preferred content. Instead of traditional therapy methods that often bore teenagers and cause them to abandon treatment, Amblyotube uses the user's own videos as stimuli, allowing them to take control of their rehabilitation. The app provides several key technical features to enhance transparency: the Dominant Eye Shader allows users to adjust blur, contrast, brightness, and opacity; the AI-Driven MFBF (Lazy Eye Sharpener) uses AI to identify human figures in the video and apply sharpening to the lazy eye's feed; and the Magenta Focus Cue, a high-visibility magenta circular signal, helps train the brain to fuse the two images. On-device inference keeps user data private and allows for offline use. The app's design choices prioritize transparency and user control, turning a traditionally opaque therapy into an auditable, user-driven process that can improve visual coordination and attention in a more sustainable and engaging manner.",
  "summary": "From Auditable LLMs to Auditable Vision Rehab: Why Transparency Wins A recent dev.to post sparked a necessary conversation about LLM-generated ratings. The author proposed a radical ruleset: fix the evidence window, show the actual quotes, admit limitations, and provide a \"can't judge\" button instead of inventing a score. The core objective is to stop confident-sounding noise from masquerading as…",
  "key_points": [
    "Guidelines proposed for LLM-generated ratings to fix evidence window and display quotes",
    "Amblyotube app uses dichoptic vision training with transparent layer over user content",
    "On-device inference and Magenta Focus Cue enhance transparency and user control"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}