{
  "id": 2440182,
  "title": "We ran our own note checker against a public scribe benchmark. Here is what it missed.",
  "url": "https://urgent.news/2026/08/21/we-ran-our-own-note-checker-against-a-public-scribe-benchmark-here-is",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-21T21:44:49.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/krasynemr/we-ran-our-own-note-checker-against-a-public-scribe-benchmark-here-is-what-it-missed-240b"
  },
  "original_language": "en",
  "account": "Krasyn's Note Check tool analyzes AI-generated clinical notes against a transcript. It labels each sentence as Supported, Unsupported, Contradicted, Scaffolding, or Unverified, and identifies three types of code flags. The clinician signs the note without edits. The results are published without accuracy figures as it hasn't been measured against a clinician-adjudicated reference set. This article presents the first findings using open data that was already labeled.",
  "summary": "Krasyn ships a tool called Note Check. Paste a visit transcript and the note any AI scribe drafted from it, and it labels each sentence Supported, Unsupported, Contradicted, Scaffolding or Unverified against the transcript, raises three pure-code flags (a number the transcript never contained, a denial about a topic raised and never denied, content filled over an inaudible marker) and lists the…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}