The Bug Class Where Your Code Is Correct, Your Tests Pass, and It Never Runs
Across one of our agents, the large majority of genuine defects - eight out of nine, when I went back and classified them - belonged to a single category. Not off-by-one, not a race, not a bad regex. The category is: the code was written correctly, it was tested correctly, and it never executed on the path that mattered. Every one of them looked healthy on a dashboard. Every one had passing unit…
Three significant software defects in the article all share a common characteristic: they were coded correctly, passed tests, yet never executed the paths that mattered due to improper configuration paths or misunderstood assumptions.
Case 1 involved a feature that at first glance appeared fine, but a toggle value (useConditioning=false) was the root cause. The code was written correctly, unit tests passed, and its presence never alerted anyone to its absence. The toggle defaulted to false, meaning the conditioned intervals never ran, and the published bands were always the unconditional ones. Test coverage was high at 84%, masking the issue as the test only verified the unconditional path.
Case 2 introduced a function that maintained internal state across invocations. In Pine Script, this is typical for moving averages and correlations. The code looked correct as it returned plausible numbers for the EMA branch, but this branch was never taken because only one of the two branches (EMA or SMA) executed per bar. The other branch was fed incomplete data, resulting in an incorrect moving average. Writing the function to compute both branches unconditionally and then select the correct one fixed the issue.
Case 3 focused on a text sanitizer function meant to convert typographic characters to ASCII. However, it contained a check that replaced " - " with " - ", effectively collapsing indentation in files. This check ran fine in "check mode", flagging false positives. Yet in "fix mode", it silently altered the files' structure. This clean-up step, designed for a specific case, became a global transformation affecting every file.
The unit tests for this function only verified the specific case where it was supposed to work, missing its negative impact on other inputs.
The key takeaway is that unit tests, which call the functions directly, do not account for the larger context of how components interact and under what conditions they execute. Reviewing the call graph, specifically identifying paths under which components run is crucial. Every default should be regarded as a decision point that can alter execution paths.
By checking observable outputs rather than internal states, and by actively tracing through all possible execution paths after adding new features, many such subtle defects could be uncovered early. In essence, written code is not always as reliable as it seems; it must be rigorously verified to ensure it runs under the conditions it was intended to.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
