My mutation score was 1.000 in two places and the denominators disagreed
I published a number, went to confirm it, and the confirmation disagreed with it. The number was fine. The confirmation was broken, and it was broken in a way I had already written a warning about. The number My evidence-bundle verifier has one metric I care about more than coverage: for every place in the code where the tool can refuse, is there a test that notices if I delete that refusal? So…
I ran a tool, expected 168 mutation sites, got 170 instead. The discrepancy arose from two different denominators. The tool measures code quality differently based on the environment it runs in. My local worktree had uncommitted changes, which altered the suite and excluded certain mutants from mutation detection. The tool's score depends on the specific state of the codebase at the time of testing.
I previously wrote a comment in my CI configuration to clarify that the score is only comparable if the suite remains unchanged. I ran a local sweep to verify a published figure, but the discrepancy highlights the importance of using the same setup for both local testing and CI runs.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
