{
  "id": 4497407,
  "title": "AI is eating website traffic, websites are blocking AI – and reliable information is getting harder to find",
  "url": "https://urgent.news/2026/08/30/ai-is-eating-website-traffic-websites-are-blocking-ai-and-reliable",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-30T20:16:47.000Z",
  "source": {
    "name": "The Conversation AU",
    "slug": "the-conversation-au",
    "url": "https://theconversation.com/ai-is-eating-website-traffic-websites-are-blocking-ai-and-reliable-information-is-getting-harder-to-find-289817"
  },
  "original_language": "en",
  "account": "The world wide web has traditionally operated on a social contract where websites allow search engines to access their content for free, in return for links back to the source. However, this arrangement is beginning to break down due to the rise of artificial intelligence (AI) tools. These AI crawlers are not just using websites to generate links, but to train models and generate answers, which may or may not be accurate. This has led to a negative cycle where website owners lose traffic and revenue, causing some to block AI scraping tools. As a result, AI results often depend more on low-quality websites, many of which are also generated by AI. This makes reliable information harder to find. Cloudflare estimates that over half of all web traffic now comes from AI bots, many of which are crawlers. While websites can use robots.txt to prevent AI crawlers, some AI companies may ignore these requests. This could lead to AI answers being based on low-quality sources, especially when it comes to misinformation. In the short term, there's a phenomenon known as \"Google Zero\" where traffic from Google drops to nothing, as AI summaries replace the need to click through to source content. This could impact site owners' revenues, as they won't receive links from AI Overviews. As a solution, some sites are considering pay-to-crawl models, but they haven't gained much traction yet. By September, Cloudflare sites will block AI crawlers by default on pages with advertising, potentially affecting up to 30% of the world's top sites in Google AI Overviews summaries. In the meantime, to find more reliable information, users are advised to scroll down and click on actual search results, as this benefits content creators and is more likely to provide accurate information.",
  "summary": "AI has broken the economic bargain behind the web, and everybody is losing.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}