A Reader Audited My OSS Release in Public. He Found the Contradictions I Missed.
When I shipped v0.2.1 of PlannerCritic , I thought the hard part was over. The engine had survived a 170-goal field test across 40 domains. The published results looked strong: 73 of 73 balanced goals approved, 96 of 97 strict goals escalated, 8 of 8 adversarial goals blocked, 30 verdict deltas all attributable, and zero new engine issues discovered by the sweep. That was the release story I…
A reader audited the author's public release of PlannerCritic v0.2.1 and found contradictions the author had missed. The reader proposed a stricter process of freezing certain claims from the release, verifying them only from public artifacts, and recording any divergences. Five claims were frozen before inspection and three of them were found to be incorrect.
Two failures were minor, one was a contradiction in the release document itself, and another was a discrepancy between the claimed number of zero true failures and the actual number of one true failure. The release documentation had errors in reporting the number of oscillation detections and the number of goals for which that condition was met.
Despite the findings, the underlying engine results were still solid and the 30 verdict deltas were attributable. The next release, v0.2.2, focused on fixing the exact class of mistakes found in the audit, resulting in a tighter evidence chain and a more trustworthy release. The key lesson was the importance of freezing claims before inspection, verifying them only from public artifacts, and recording partial confirmations honestly to prevent trust from eroding due to overstated or inconsistent release notes.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
