What I learned checking 65 client domains with RDAP instead of WHOIS
I built a domain expiry monitor, and to test it against something other than my own domains I pointed it at the "our work" pages of 30 US web design agencies — the client sites they link to publicly — and checked every domain against its registry. The results were mildly alarming, but the interesting part was everything that went wrong on the way there. The results first 65 client domains across…
I conducted a test on 65 client domains using RDAP instead of WHOIS, observing that one domain had already expired, while 60% of the rest had expirations within 3 to 6 months. The challenging aspect was the differences in data formats between WHOIS and RDAP, with RDAP returning structured JSON responses over HTTPS, and 404 errors indicating unregistered domains.
Three issues arose during the process: 1) the registrar's name is concealed within a jCard, 2) IANA's bootstrap lacks coverage for all TLDs, and 3) public suffixes can mislead with incorrect expiry dates. Additionally, my initial approach of checking every href on the page led to false positives, which was resolved by using a structural rule to only consider anchor tags. Lastly, approximately one-third of websites blocked automated requests, prompting a switch to using a real browser to overcome IP reputation blocks.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
