0 of 550 top homepages send noai. 21 send Bing's noarchive.
I fetched the homepages of the Tranco top 1,000 and read every meta robots tag and every X-Robots-Tag header they sent. 550 homepages came back as real pages. None of the 550 sends noai or noimageai . None sends nosnippet either, and the 47 that send max-snippet all set it to -1 , which means "no limit". The page-level AI rule that big sites actually use is Bing's: 21 of the 550 send noarchive or…
In a study of the top 550 webpages, it was found that none of them send directives related to noai or noimageai, nor do they use nosnippet. Out of the 550, only 21 send either noarchive or nocache, which are signals to prevent pages from appearing in Bing Chat answers or to limit their use in training. This suggests that the primary AI policy for these top websites is primarily managed through the robots.txt file.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
