{
  "id": 10118344,
  "title": "My prompt-injection fix caught 0 of 20 attacks. The part I almost didn't build caught all of them.",
  "url": "https://urgent.news/2026/09/27/my-prompt-injection-fix-caught-0-of-20-attacks-the-part-i-almost",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-27T02:45:52.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/vishalhabib99/my-prompt-injection-fix-caught-0-of-20-attacks-the-part-i-almost-didnt-build-caught-all-of-them-oi0"
  },
  "original_language": "en",
  "account": "The reporter built a checker for AI-drafted answers to retirement questions, called \"retirement-answer-check\". This system decides whether to send or review a draft based on code checks and two model judges handling non-numeric wrong facts, personal advice, and promises about returns. The reporter tested the system by creating 20 attacks, with 12 real planted problems and 4 correct answers with injected text. The first round of attacks showed that the system failed to catch every planted problem, despite the judges flagging them. However, 3 out of 4 injected drafts went straight to the customer. The reporter then introduced fail-closed plumbing, which forces any text aimed at the checker to be flagged and sent to review. After implementing this fix, the regex caught all 16 attacks in the second round. The article concludes that the system still needs testing for new attack methods and that a defense written while looking at the attacks proves nothing until someone new attacks it.",
  "summary": "I built a checker for AI-drafted answers to retirement questions ( retirement-answer-check ). Before a customer sees a draft, it decides SEND or REVIEW. Plain code checks every number against an IRS-sourced facts table. Two model \"judges\" handle what code can't read: non-numeric wrong facts (\"yes, that exception covers your 401(k)\"), personal advice, and promises about returns. When I wrote the…",
  "key_points": [
    "Retirement-answer-check system failed to catch all 20 attack prompts",
    "Fail-closed plumbing introduced, flagged all 16 attacks in second round",
    "System needs further testing for new attack methods"
  ],
  "editors_take": "Adding a fail-closed mechanism to force flagged text to review significantly improved the system's ability to catch malicious inputs, highlighting the importance of a previously considered secondary component.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}