git bisect run named the wrong commit 10 out of 10 times — so I gave every commit its own clean room
git bisect run is the right algorithm. What I did not expect is how confidently it lies when you leave it running unattended. I generated a 255-commit Python history in which a dependency pin ( pytz ) moves from 2022.1 to 2023.3 at commit 150, together with the golden value of the failing test. Each commit's test is right under its own pin and wrong under the other. A real bug is planted at…
At first glance, git bisect run appears to be a reliable algorithm; however, the author discovered that it consistently makes incorrect judgments when the searched range contains a moving dependency pin. This was demonstrated through an artificial repository with 255 commits, where a dependency (pytz) shifted from version 2022.1 to 2023.3 at commit 150, accompanied by a failing test.
The test corrected itself at commit 173, which was the planted commit with a bug. When the author ran git bisect run on this repository ten times, all results were wrong, indicating that git bisect trusts a single environment for every commit. This flaw allows a moving pin to lead the search into the wrong half, resulting in false positive convictions.
The author concludes that to avoid such issues, each suspect commit should have its own clean environment for testing.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
