{
  "id": 9098889,
  "title": "We Thought the LLM Was Wrong. Our Safety Detector Was Wrong.",
  "url": "https://urgent.news/2026/09/22/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-22T06:35:19.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/iamwaqarjaved/we-thought-the-llm-was-wrong-our-safety-detector-was-wrong-50pc"
  },
  "original_language": "en",
  "account": "Our investigation into LLM safety benchmarks revealed an unforeseen issue: the safety detector itself can be faulty. While evaluating an open-source AI security framework, we noticed seemingly inconsistent model behavior. This led us to uncover several problems, including Unicode normalization failures, incomplete refusal vocabularies, and cross-model differences. However, the most alarming finding was false PASS classifications introduced by our own detector improvement. This false PASS was concerning because uncertainty is visible, unlike a genuine failure. For a comprehensive look at our findings, visit https://agentsafelabs.com/blog/we-thought-wed-found-a-model-bug-wed-actually-found-a-detector-bug/. The open-source implementation is available at https://github.com/AgentSafeLabs/safelabs-eval. I am keen on learning how other developers approach testing the classifiers they employ to assess LLM behavior.",
  "summary": "There is a hidden dependency in a lot of LLM safety benchmarks: the detector. You send an adversarial prompt to a model, collect its response, and then some classifier decides whether that response represents refusal, compliance, or ambiguity. Eventually those classifications become percentages in a safety report. But what happens when the detector itself is wrong? We encountered exactly that…",
  "key_points": [
    "Safety detector may be faulty, not the LLM",
    "False PASS classifications introduced by detector",
    "Comprehensive findings available at agentsafelabs.com"
  ],
  "editors_take": "The discovery that a safety detector can be faulty undermines confidence in current methods for evaluating LLM behavior and highlights the need for more robust testing of classifiers.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}