My prompt-injection fix caught 0 of 20 attacks. The part I almost didn't build caught all of them.

The reporter built a checker for AI-drafted answers to retirement questions, called "retirement-answer-check". This system decides whether to send or review a draft based on code checks and two model judges handling non-numeric wrong facts, personal advice, and promises about returns. The reporter tested the system by creating 20 attacks, with 12 real planted problems and 4 correct answers with injected text.

The first round of attacks showed that the system failed to catch every planted problem, despite the judges flagging them. However, 3 out of 4 injected drafts went straight to the customer. The reporter then introduced fail-closed plumbing, which forces any text aimed at the checker to be flagged and sent to review. After implementing this fix, the regex caught all 16 attacks in the second round.

The article concludes that the system still needs testing for new attack methods and that a defense written while looking at the attacks proves nothing until someone new attacks it.