Your prompt system has no tests, and that is why you cannot tell it is broken

A multi-skill agent system, comprised of 15 skills and nine commands, was designed to write structured JSON. It functioned for weeks without any issues before a subtle failure went unnoticed. The failure occurred when one skill stopped writing a single field, leading to the next skill reading a null value and continuing as usual.

The resulting score was slightly lower than expected, but the output appeared convincing to the user. The problem lies in the fact that these AI models are exceptionally good at producing plausible outputs, even when broken.

Testing proved challenging due to the nature of AI outputs. Running the same prompt twice yields different results, which is expected behavior and acceptable. However, the real test lies in other factors. Three key aspects were found to be testable, collectively catching nearly all regressions. First, the arithmetic - a penalizing system was implemented to score six weighted dimensions, allowing for easy detection of any dimension falling below a certain threshold.

Second, the shape - every skill writes to a file with an expected structure, including required fields, enums, nullable values, and conditional requirements. This schema validation caught more real regressions than any other code written. Lastly, the prose rules that are actually numbers - specific constraints were placed on memo writing, such as word budget, required sections, and citations of URLs.

The checker, a Python script with approximately 750 lines and no dependencies, was designed to run on every file write inside a hook. It checks the output against predefined directives like required fields, enums, nullable values, and more. The contract is stored in one JSON file keyed by output filename. To ensure the accuracy of the checker, it was mutation-tested by introducing specific breaks and confirming the harness caught them. This process revealed that one of the author's own tests was passing for the wrong reason.

The checker acts as the sole barrier between the system and "confident nonsense." It is essential to test the tests themselves. In a system where output generation is involved, the checker is the only component that ensures confident, accurate results. Wiring the agent to fix its own output without human intervention is also crucial.

Claude Code's PostToolUse hook, executed after a file write, runs the checker on the affected folder and exits with an error code if any issues are found. This error code is communicated back to the model, which can then correct the file before proceeding. This self-correcting mechanism, implemented in under 40 lines of code, is essential for a reliable testing system.