How long do you keep a published claim provisional?
I retracted a benchmark this week. Two weeks ago I published a timing figure, described it as a fixed cost, and built an argument on it. Today the same script on the same machine ran two and a half times faster, and the reason was that I had taken the original reading while my laptop was under a load average of 70 from an unrelated experiment. That is the fourth thing I have had to correct this…
The author of a published claim faces a decision on how long to retain the validity of their findings. They have experienced four instances of needing to correct published figures, each due to factors like unrecorded conditions, truncation of queries, retry helper budget issues, and reader feedback. The author believes the optimal approach is to treat all published numbers as provisional and version the post with a changelog.
However, this would require maintaining multiple versions of posts, which they find unsustainable with their current volume. Alternatives include setting an expiry date for measurements, which they find cheap but insufficient for catching immediate errors. The author also observes that corrections often lead to more useful content than the original posts.
They question whether their writing team or their own practice has an explicit shelf life for measured claims, or if they only retain truth until they receive feedback. The dilemma lies in whether to make numbers permanent until challenged or to maintain a flexible approach to revisiting published information.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
