{
  "id": 14295240,
  "title": "Your AI review bot is training you to ignore it",
  "url": "https://urgent.news/2026/10/13/your-ai-review-bot-is-training-you-to-ignore-it",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-13T21:24:50.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/jeff_pdc/your-ai-review-bot-is-training-you-to-ignore-it-23c5"
  },
  "original_language": "en",
  "account": "A year ago, an AI reviewer was enabled for every pull request. However, after a month, most of it was turned off. This wasn't due to the AI being incorrect, but because the comments it generated were noisy, and noise can make a review tool ineffective. Week one saw everyone reading every comment. By week three, the bot flagged trivial issues like formatting nits and nulls. By week six, users began scrolling past the comments to check if the merge was blocked. This illustrates the trap of a noisy review bot. The AI is not deceiving users; it's simply flagging every check it knows, many of which have low precision. As a result, the signal-to-noise ratio deteriorates, and the comments become unimportant. High precision is more important for a review bot than high recall. A bot that is right 80% of the time gets read, while a bot that is right 40% of the time becomes background noise. The solution is to lower the confidence threshold for low-confidence rules. For instance, style nits should be checked by a linter, not a PR comment, and speculation about intent should be in a suggestion, not a blocking thread. The bot should be quiet by default and loud only when it has high-confidence findings. Where these high-confidence findings reside are the ones with definitive answers, not just \"this might be unsafe\" but \"the code returns 200 but the contract says 201.\" These are facts that can be checked against machine-readable sources, and they have a low false-positive rate. Therefore, when the bot speaks, it should always mean it. The high-confidence checks include API changes like contract drift, which can be run locally and are difficult for humans to overlook. To improve the review process, one should analyze the review bot's history to count how often a human agreed with or ignored its comments. Rules with the lowest agreement should be turned off first. This will result in a bot that comments on one out of five PRs instead of every PR, and when it does, people will read it. A reviewer that nobody reads is worse than no reviewer at all, as it creates a false sense of coverage.",
  "summary": "A year ago we turned on an AI reviewer for every PR. Last month we turned most of it off. Not because it was wrong — because it was noisy, and noise is how a review tool loses its teeth. The decay is predictable Week one: everyone reads every comment. Week three: the bot flags a formatting nit on line 41, a possible null on line 88, and a question about a variable name. Two of three are…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}