I asked an AI to find bugs and got 0. Changing one line of the prompt found 5.

You may have heard that AI bug checks cannot be trusted because they often say "no issues found." However, I built apps with AI and observed the number of bugs found change from 0 to 5 with the same code and model - the only modification being one line in the prompt. In another instance, an AI detected a significant bug in a calculator that a rule-based checker had previously reported as having 0 issues.

My investigation into what Fable excels at revealed that its strong point is persistence. I reasoned that it might be ideal for repetitive, monotonous tasks such as identifying and fixing bugs. To test this hypothesis, I asked Claude Code whether it made sense, and it agreed that the approach was reasonable. I proceeded to conduct an experiment.

After a month, I compared Fable's performance with other models using the same prompt for 12 apps containing real bugs. The results showed that Fable found 7 bugs with only 1 app attempted, while Haiku found 4 out of 12, and Sonnet found an equal number of 4 out of 12 apps. More expensive models unfortunately performed worse, only finding half of the bugs. This suggests that leaving AI bug checking to cheaper models can leave more than half of the bugs undetected.

Fable is relatively pricier, costing around ¥80-145 per run, which translates to less than $1 per run. However, I found it beneficial to use it as the final step in the bug-checking process rather than on every change. The setup involved using Claude Fable on a version of my image mosaic blur app that still had real bugs present. I had removed git history to focus on the actual bugs that slipped in during development.

In the first run, Fable returned 0 bugs after comparing every file and proving that no intentional changes were made. However, the real bugs remained undetected. For the second run, I modified the prompt to "Find the holes still left in this version and close them," and Fable successfully identified 5 real bugs: right-clicking drawing unintended dots, second finger input overwriting touch input, loading another image during a drag causing unwanted lines, mismatched image name in the spec, and a large image loading issue resulting in a misleading error message. Each bug was fixed, and Fable provided detailed explanations for each one.