Runbook Rot: Three Fingerprints of Stale CI Docs and the Check That Catches Each

Three distinct fingerprints reveal when Continuous Integration (CI) documentation becomes outdated and potentially hazardous. The first fingerprint, timestamp gap, indicates that a document's last meaningful edit is older than the last change to the subject it describes. This is not simply a matter of file modification timestamps; it involves comparing the last content change of the document against the last change of the subject it documents.

The second fingerprint, orphaned referent, occurs when a runbook names elements such as job names, secret names, or service names, but these named elements no longer exist within the pipeline. This can lead to a map of a neighborhood with one building demolished, as the document remains internally consistent but points to something that no longer exists.

The third and final fingerprint, procedural verb, involves stale documents outsourcing living processes. These are marked by verbs with no clear object that can be executed, such as "ask Alice about the staging cert" or "ping the platform team." The presence of these procedural verbs can lead to tribal knowledge ticking faster than any referent, resulting in a nudge becoming a five-minute outage.

To address these issues, the author proposes auditing CI documentation against these three fingerprints using mechanical checks integrated into the CI pipeline. By treating runbooks as rows of data rather than pieces of writing, the author argues that stale documentation can be detected and updated proactively.