{
  "id": 2124807,
  "title": "Don't trust model quotes; use anchors instead",
  "url": "https://urgent.news/2026/08/20/dont-trust-model-quotes-use-anchors-instead",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-20T11:19:24.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/gmhoward9289ops/dont-trust-model-quotes-use-anchors-instead-2ohl"
  },
  "original_language": "en",
  "account": "When I asked artificial intelligence models to extract information with sources, the results varied widely. The naive approach, which simply returned the model's quote, matched 37–93% of the time. However, the fabricated quotes were real, ranging from 0–13%. Normalizing the quotes improved the accuracy slightly. In contrast, the anchor method returned a short phrase near the extracted value and used deterministic code to search the source. This approach found the exact match, then normalized Unicode and whitespace, searched for ordered token subsequences, and performed fuzzy matches. If the anchor was found, the containing sentence from the source was emitted; otherwise, \"not_found\" was returned. The anchor coverage was a solid 93–100%. This method achieved 93–100% anchor coverage, while the quote method only quoted correctly 37% of the time and still recovered 93% of anchor coverage. The source material was crafted to expose the flaws in quote-based extraction, including curly quotes, en-dashes, non-breaking spaces, double spaces, typos, parentheticals, version strings, numeric confidence intervals, and more. By using anchors, the system never produces made-up spans because it does not create sentences; it merely points to existing text. A model that occasionally quotes incorrectly is worse than one that admits it cannot find the quote. The anchor system forces you to verify the findings, ensuring that you do not rely on potentially inaccurate quotes. When I considered the consequences of using the quote method, I realized that even a system with a high accuracy rate could produce unreliable results, leading to a flawed pipeline. If I had relied solely on the quote method, I would have built a pipeline with the same strengths and weaknesses as its best days, despite occasional errors. Anchors provide a more reliable method by eliminating the risk of fabricated spans. The key difference between extraction accuracy and reliability is whether the provenance can lie. Ensuring provenance integrity is critical when citing information later.",
  "summary": "I asked models to extract facts with provenance two different ways. Naive: return {answer, quote} , insist the quote is verbatim, then string-match it against the source. Exact-match rates ran 37–93% depending on model quality. Fabrication was real — 0–13% . Normalization helped only slightly. Every tier could produce prose that reads like a quote and isn't. Anchor: return {answer, anchor_phrase}…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}