Why "works on my machine" often means "I'm still in variant B" — QA rules for testing behind A/B experiments and feature flags
If your product ships behind A/B experiments and feature flags, your QA team has a failure mode nobody tracks: variant amnesia . You add yourself to an experiment's testing group to check a feature, switch variants a couple of times, move on. A week later you're testing something else, the app behaves strangely, and you file a bug — not knowing you're still enrolled in variant B of an experiment…
When your product is released behind A/B experiments and feature flags, a common issue arises: variant amnesia. As a QA team member, you may add yourself to an experiment's testing group to test a feature, switch between variants, and then move on to another task. Several weeks later, while testing a different aspect of the product, you may encounter unexpected behavior.
This strange behavior leads to filing a bug report, unaware that you are still enrolled in variant B of an experiment you have forgotten about. Lead QA engineer explains the costs of this variant amnesia, the rules implemented to combat it, and the tool they built to streamline the process.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
