133 of 10,099 Shopify stores block an AI crawler. Six block the one ChatGPT shops with.
Merchants are told two opposite things about AI crawlers: block them, because they take your content and give nothing back; and admit them, because that is how a store gets into an AI shopping answer. Both assume a decision is being made. We wanted to know how many stores have made one, and which way. Method Every store in a corpus of 10,099 known Shopify storefronts has its /robots.txt read as…
A study of 10,099 Shopify stores found that 133 of them block at least one AI crawler, amounting to 1.32% of the total. The majority of these blocks, 120, are aimed at training and grounding crawlers, which feed into AI shopping answers. Only 13 stores block crawlers that actually fetch data in response to user queries. Among the blocked crawlers, OpenAI's GPTBot is the most commonly blocked, followed by OAI-SearchBot, used for ChatGPT search and shopping results.
Notably, 6 of the 8 stores that utilize the noai or noimageai meta robots tag on their product pages do not block any AI crawlers either. This suggests that the tag, while intended to opt out of training, does not necessarily extend to shopping crawlers, based on the study's findings.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.