Almost every crawl error on our domain came from a site that didn't exist yet
Originally published on the WatchNext blog . Google had crawled WatchNext and indexed none of it. The coverage report listed 11 pages as crawled, currently not indexed and 200 as discovered, currently not indexed — found, but never fetched. The crawl stats report had what looked like the reason. Of 453 crawl requests over the month, 17% had ended in a DNS error. If Google couldn't reliably…
Google had crawled WatchNext and indexed none of its pages. The coverage report showed 11 pages as crawled, currently not indexed, and 200 as discovered, currently not indexed. The crawl stats report indicated 17% of 453 crawl requests over the month had ended in a DNS error. The problem wasn't with WatchNext, but with the DNS errors occurring for four unrelated sites hosted at different subdomains under leyu.studio.
The issue arose because Google couldn't resolve leyu.studio, resulting in 77 failed requests, which accounted for the 17% DNS error rate. The apparent match between the 8.17% HTML requests and the 11 crawled pages was a coincidence, as the ratio didn't accurately represent the WatchNext app's structure. The fix involved resolving leyu.studio and adding a separate URL-prefix property for each site, isolating the WatchNext crawl report to focus on its performance.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
