Half the Requests Wearing ChatGPT's Name Were Scanning Us for Secrets

Cloudflare launched AI Crawl Control, allowing them to monitor AI agents visiting their website. In the initial 24-hour period, they observed 1,890 requests from AI agents, with 483 of those requests failing. This failure rate increased by 63 percent when compared to the previous day. Among the agents they were most interested in was ChatGPT-User, which was documented as user-initiated.

It accounted for 859 requests, but only 412 of those returned a 200 status code, meaning roughly half of the requests with this name were failing. Initial investigation suggested a redirect problem due to recent migrations of document slugs, but further examination revealed a vulnerability scanner's list of items to probe: SSH keys, Terraform state, environment backups, CI configuration, and more.

All 447 failures had the user agent ChatGPT-User. While this may seem like a common occurrence for any public site, the distinction lay in the source of these requests. The label of ChatGPT-User was misleading, as it only identified traffic based on its user agent string, not the actual sender. To determine if these requests were coming from OpenAI, users could refer to OpenAI's published IP ranges or use Cloudflare's cf.client.bot field in their WAF custom rules.

Additionally, ChatGPT serves pages from a shared cache, meaning even if a request comes from a ChatGPT-User agent, it may not directly interact with the user's server. This story began as an investigation into an alarming AI metric but ultimately exposed the need for more accurate identification of AI traffic and the potential security risks associated with vulnerable scanning.