The Half-Life Layer — Or, "It Passed in March" Is Not the Same as "It Passes Now"
Somewhere in your repository there is a test that passed in March. It passed against a schema that has since gained two columns, a dependency that has since changed its default timeout, and a traffic profile that has since shifted from mostly reads to mostly writes. The test still passes. Nobody has run it against the current system in a way that would notice the difference. That is not a broken…
A test that once passed in March may not pass today due to changes in the system it was built to test. These changes can be in the schema, dependencies, traffic profile, or population. The test itself does not become wrong, but the measurement it provides may no longer reflect the current system. This issue, known as the "half-life layer," arises when evidence is cited without specifying the timestamp of when it was valid.
There are three ways evidence can expire: changes in the substrate (system structure), population (data or user behavior), or witness (people responsible for observing). The problem is that changes often leave no trace in version control, and the test can still pass while the system it was built to test has moved on. To prevent this, it is essential to explicitly state the assumptions and conditions under which a test or claim is made, and to re-verify these assumptions regularly.
A "five-column evidence register" can help with this: one row per claim, with columns for the artifact, the measured conditions, when it expires, and who is responsible for re-verifying it. This approach ensures that claims are falsifiable and tied to specific events, rather than arbitrary dates.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.