Tailscale says deeply buried 16-year-old SQLite bug caused last year's outages
Think your bugs are a pain to track? WAL-Reset took six months of hunting and development of a new logging tool to identify
Tailscale, a peer-to-peer networking platform, experienced outages earlier this year due to a 16-year-old SQLite bug. The issue, known as the WAL-Reset bug, was discovered after six months of investigation. SQLite, which Tailscale has used since 2022, stores all information about the virtual private mesh networks (tailnets) it hosts.
Tailscale's backup pipeline takes a snapshot of the database every few minutes and uploads it to an S3 bucket. However, corruption was repeatedly detected in these backups starting in August 2025. Despite numerous attempts to reproduce the issue, the Tailscale team couldn't find any common triggers. They eventually turned their attention to SQLite's checkpointing process, which moves new database entries into the master database file.
SQLite's Write-Ahead Log (WAL) is used to improve performance and concurrency. The WAL is checkpointed in a process invisible to end-users, but Tailscale manually controlled this process for faster and consistent backups. The SQLite team developed a new tool to log checkpointing activity, revealing the WAL-Reset bug. This rare data race occurs when a write occurs during a checkpoint, confusing the checkpointing process and causing permanent data loss.
SQLite has known about this bug since version 3.7.0, released in 2010, but it remains extremely unlikely to happen in ordinary use.
Written by urgent.news from The Register Software's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
