I Wrote 238 Tests Against My Own Auth Package and Found 4 Real Bugs
I'd already done a lot right by the time I started writing tests for Beaver-Auth . Every module had gone through multiple rounds of deliberate review. Enumeration protection, hashed tokens, refresh rotation, TOTP replay defense — the design was solid, and I knew it was solid, because I'd thought hard about every piece of it. Then I wrote 238 tests against the actual code, and found 8 real bugs.…
The author had already put Beaver-Auth through multiple rounds of review, with a solid design incorporating features like enumeration protection and TOTP replay defense. Despite the thorough review, the author discovered 8 real bugs after writing 238 tests against the actual code. The issue lies in the gap between "I reviewed this carefully" and "this is shippable," which is often larger than assumed, even with careful review.
The author built a solid test suite covering registration, login, and verification processes. Each test passed, leading to a feeling of completion. However, passing tests only verify that the code does what the tests expect. If the tests were written from the same mental model as the code, they could confirm a bug as correct behavior because both the code and the test would agree on the same wrong assumption.
Bug 1 involved TypeScript allowing an argument-shift bug to compile clean. The TaskDispatcher interface had its signature drifted, missing a payload parameter. The default implementation still expected four arguments, causing the payload object to be placed in the handler slot, leading to a TypeError: handler is not a function. This bug could silently break production emails.
Bug 2 illustrated a feature that appeared complete but had no way to be called. The LoginEngine had a logoutJwtSession(familyId) method, but no way to obtain the familyId from the public API. This issue could only be found through integration testing, where the login response was role-played to access the logout feature.
Bug 3 highlighted an error type leakage. ValidationEngine was documented to only throw ValidationError, but it delegated to an internal NormalizationError used for resource-limit violations. The NormalizationError was never wrapped, allowing it to propagate as an internal error type. This could lead to incorrect error handling in consumers following the documented pattern.
These bugs were not caught by simply writing more tests, but by testing against the real, integrated system and the actual call chains. The author emphasizes that passing tests and being shippable are different claims, with a gap that many overlook even when reviewing is genuinely careful.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
