{
  "id": 3908607,
  "title": "The linter ecosystem is actively migrating silent failure into an undetectable idiom",
  "url": "https://urgent.news/2026/08/28/the-linter-ecosystem-is-actively-migrating-silent-failure-into-an",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-28T06:17:57.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/feiiiiii5/the-linter-ecosystem-is-actively-migrating-silent-failure-into-an-undetectable-idiom-2h70"
  },
  "original_language": "en",
  "account": "A growing trend in the linter ecosystem is transforming silent failures into undetectable patterns. In the case of LLM evaluation bugs, a judge model may fail without any noticeable impact on the benchmark scores, as a handler turns an error into a legitimate zero value. This silent failure pattern has been observed repeatedly in various mainstream AI/ML open source projects. To address this issue, the author built a detector called failroute, a static analyzer that flags such failure-to-success conversions. failroute is available on PyPI, integrates with CI pipelines, and its findings are reproducible from the source code. The author found that the modern idiom for ignoring failures in Python has evolved from explicit try-except-pass statements to using contextlib.suppress, which discards exceptions in a semantically identical manner. Surprisingly, no shipped linter currently flags the use of contextlib.suppress. In fact, Ruff's SIM105 rule even recommends rewriting try-except-pass constructs into suppress blocks, resulting in a widening of the detection gap. The author highlights that while using contextlib.suppress is a valid approach in certain scenarios, it poses a challenge for detection tools as they cannot identify this pattern. The author emphasizes the importance of manually reviewing and deciding whether discarding specific failures is appropriate for each case. The study found that out of 613 total findings in AI/eval repositories, 77 suppress blocks were present alongside 403 silent-fallback/masked-exception handlers, indicating a significant migration of silent failures into an undetectable form. The author concludes that as we rely more on AI assistance in building detection tools, it is crucial to have a deterministic gate, such as a hand-labelled corpus, to ensure the correctness and reliability of the rules. The full process of building such a corpus is documented in the repository's process doc.",
  "summary": "In a well-known family of LLM evaluation bugs, the failure doesn't look like a failure. A judge model goes down, and the benchmark keeps publishing scores — because somewhere in the code, a handler like this turned an outage into a legitimate-looking zero: try : score = await llm_judge ( prompt ) except Exception : return 0.0 I hit this pattern repeatedly while fixing correctness bugs across…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}