DNS change monitoring: the seven false alarms I had to kill

Designing an effective DNS change monitoring system involves avoiding common pitfalls that can lead to false alarms. A basic DNS checker consists of just ten lines of code, which involves looking up a record, comparing it to a stored value, and sending an email if they differ. However, DNS answers can change in unexpected ways, leading to false positives. The author of the DNS Notify checker faced seven specific issues that required careful handling to prevent these false alarms.

Firstly, using a system resolver to look up records can lead to false positives due to cache variations. Instead, querying the domain's own nameservers directly avoids this issue. Secondly, DNS servers may rotate multi-value answers, such as A records, which can cause a naive comparison to flag a change when none has occurred. Sorting the values before comparison resolves this.

Thirdly, Time to Live (TTL) values should not be included in the comparison to prevent false alarms caused by the TTL counting down from a recursive resolver or being reset by an authoritative server. Fourthly, the Serial Number in the SOA record should be masked before comparison since updates to individual records may increment the serial number without any true change.

Fifthly, timeouts that appear as deletions need to be handled separately. A timeout or SERVFAIL is a failed check and should not be treated as a deletion.

Sixthly, formatting differences such as trailing dots in hostnames or split TXT records need to be canonicalized to ensure consistent comparisons. And finally, false alarms can occur when a record changes and then is promptly corrected. Keeping track of both the accepted value and the current value allows for distinguishing between genuine changes and temporary corrections.

By addressing these issues, the DNS Notify checker provides reliable alerts for actual DNS changes, making it a practical tool for monitoring DNS changes without generating unnecessary noise.