I Published Every Flaw My Safety Tool Can't Catch. It Made It More Credible, Not Less.

This article is a follow-up to the PlannerCritic series, focusing on three flaws that the author was unable to eliminate in their own safety tool. The open-source tool claims to be bulletproof, but the author argues that being transparent about its weaknesses makes it more trustworthy than a tool that claims to be perfect. The flaws are categorized as Seam 1, Seam 2, and Seam 3.

Seam 1 deals with indirect injection through tool outputs. The author's testing showed that the payload could bypass the deterministic gates if it arrived after the goal text was audited. A commenter suggested that re-auditing every tool result with another LLM would solve this issue, but the author prefers the idea of tools returning typed data plus provenance, so that deterministic policy can decide whether the source may influence a particular state transition.

Seam 2 concerns well-formed malicious plans that defeat structural checks. The deterministic gates check for structural completeness, but an attacker can craft a plan with a dummy rollback and verification step, satisfying the linter while carrying malicious actions. The gates pass these plans as structurally sound. The author suggests that the critic, being an LLM, can catch semantic malice sometimes, but this is only partially closeable.

Seam 3 is the most challenging flaw. The author relies on an LLM to catch adversarial intent, but sophisticated jailbreaks can blind-spot even an adversarial system prompt. The architecture works because it does not bet the security contract on the LLM being clever, but the semantic layer alone is not sufficient. The author acknowledges that they can measure one direction of this flaw but not the other, as the critic is maximally non-deterministic yet never lets a seeded defect through.

The author believes that publishing these flaws is the credibility move rather than a weakness. By openly discussing the limitations of their safety tool, the author demonstrates engineering and transparency, which builds trust in their work.