{
  "id": 9473825,
  "title": "Breaking CI Guards on Purpose to Prove They Can Fail",
  "url": "https://urgent.news/2026/09/24/breaking-ci-guards-on-purpose-to-prove-they-can-fail",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-24T03:00:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/megapixel99/breaking-ci-guards-on-purpose-to-prove-they-can-fail-271k"
  },
  "original_language": "en",
  "account": "Breaking CI Guards on Purpose to Prove They Can Fail\n\nCI guards, although designed to prevent failures, can be deliberately broken to demonstrate their limitations. Certain types of guards, such as those covering YAML, Terraform, Dockerfiles, or other file types lacking mutation functions, are not easily covered by tools like Stryker mutation testing. The canfail tool addresses this by allowing users to declare breaks, specifying a file, anchor string, replacement, and the expected failure.\n\nTo use canfail, it is necessary to install the package via pip and then create a configuration file, canfail.json, containing the breaks to be applied. The tool first runs the check on a clean tree to establish a baseline - if the check is already red, it provides no valuable information when a break is applied. A failure must occur for the specific reason declared in the break; otherwise, it is considered a \"wrong-failure\". The anchor must match exactly once, and the file must be verified by digest to ensure restoration took place.\n\nOne key aspect of canfail is its reliance on file mtime to determine whether a break was effective. However, this approach can be unreliable for sub-second edits, as changes made to the file may not be reflected immediately. The solution was to disable bytecode caching (PYTHONDONTWRITEBYTECODE), as edits to the file would result in stale bytecode being executed, thus failing to trigger the intended failure.\n\nThe paragraph above highlights the importance of accurately detecting and reporting on the failures caused by these breaks. If a check fails for the wrong reason or does not run at all, it is not considered a valid failure. The canfail tool includes tests to ensure that it correctly identifies these scenarios, as well as to validate its own configuration and prevent silent failures or incorrect reporting. Overall, canfail provides a valuable tool for developers to understand the limitations of their CI guards and improve their configurations accordingly.",
  "summary": "Code: Megapixel99/canfail A CI guard that has never failed may be incapable of failing. A lint rule disabled by a config merge, a type check whose glob stopped matching, a schema validation step pointed at the wrong directory, a security scanner with an empty ruleset. Every one of them is green forever, and green is what you were looking for. For code guarded by tests, mutation tools like Stryker…",
  "key_points": [
    "CI guards can be deliberately broken to test their limitations",
    "canfail tool allows users to declare breaks in specific files",
    "canfail relies on file mtime to detect successful breaks"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}