{
  "id": 17833382,
  "title": "Prepare web content for Amazon Bedrock Knowledge Bases with Crawl4AI",
  "url": "https://urgent.news/2026/11/02/prepare-web-content-for-amazon-bedrock-knowledge-bases-with-crawl4ai",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-02T17:11:21.000Z",
  "source": {
    "name": "AWS Machine Learning",
    "slug": "aws-machine-learning",
    "url": "https://aws.amazon.com/blogs/machine-learning/prepare-web-content-for-amazon-bedrock-knowledge-bases-with-crawl4ai/"
  },
  "original_language": "en",
  "account": "When constructing Retrieval Augmented Generation (RAG) applications, the quality of answers relies heavily on the data that is ingested. Amazon Bedrock Knowledge Bases is a fully managed Retrieval Augmented Generation capability within Amazon Bedrock. This service takes care of ingestion, storage, indexing, and retrieval for you by connecting your data sources. You don't have to provision or scale a vector store, and embedding and reranking are performed using service-managed models by default. If you are working with web content, begin by using the Amazon Bedrock Web Crawler data source connector for managed knowledge bases. Provide this connector with seed URLs or sitemaps that you are authorized to crawl, and it will crawl within the scope and rate limits that you set. The crawler automatically keeps your knowledge base up-to-date through scheduled syncs. The connector can also wait for content that loads with JavaScript, eliminating the need for you to build, host, or operate a crawler.",
  "summary": "This post shows how to use Crawl4AI, an open source web crawler, as a content preparation layer for Amazon Bedrock Knowledge Bases, handling JavaScript rendering, selective extraction, and custom metadata enrichment before web content reaches an Amazon S3 data source.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}