{
  "id": 8107662,
  "title": "Cloudflare Blocks AI Agents From Ad-Supported Pages",
  "url": "https://urgent.news/2026/09/17/cloudflare-blocks-ai-agents-from-ad-supported-pages",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-17T21:29:38.000Z",
  "source": {
    "name": "PYMNTS",
    "slug": "pymnts",
    "url": "https://www.pymnts.com/news/artificial-intelligence/2026/cloudflare-blocks-ai-agents-from-ad-supported-pages/"
  },
  "original_language": "en",
  "account": "Cloudflare has begun blocking AI agents by default on pages that contain advertisements, which creates a conflict between the tools consumers use for product comparisons and the websites those tools rely on. The new default setting allows search crawlers, but it blocks AI training agents and user-directed agents that access any page with an ad. There is no current option for agents to opt-out of the block. Cloudflare handles a significant portion of web traffic, about a fifth of it, according to TechCrunch. The company's reasoning is that ads indicate a page was intended for human viewing; AI agents that retrieve pages on behalf of shoppers do not see these ads. Cloudflare aims to reduce mixed-use crawler traffic to zero within a year and allows site owners to control content visibility in AI summaries by mid-2027. While retailers like Amazon, Walmart, and Target are testing their own strategies regarding AI agents, Cloudflare caters to both sides of the issue, recently launching Kitesurf, a cloud-hosted browser designed for AI agents.",
  "summary": "Cloudflare has started blocking artificial intelligence agents by default on pages that carry ads, putting the tools consumers use to compare products in direct conflict with the sites those tools need to read. The new preset for ad-supported domains allows search crawlers, refuses AI training and blocks user-directed agents on any page that shows an […] The post Cloudflare Blocks AI Agents From…",
  "key_points": [
    "Cloudflare blocks AI agents on ad-supported pages by default.",
    "Company handles about a fifth of global web traffic.",
    "Cloudflare aims to eliminate mixed-use crawler traffic within a year."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}