Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK

This post demonstrates building enterprise agentic retrieval on a Managed Amazon Bedrock Knowledge Base using AWS Cloud Development Kit (AWS CDK). To understand the importance of this, it's crucial to consider where ordinary Retrieval Augmented Generation (RAG) systems fall short. Most RAG systems follow a consistent pattern: they embed the question, fetch the nearest chunks from a single index, pass them to a model, and return the answer.

For many questions, this design works well, but it fails in certain scenarios. If a question requires information from multiple corpora, a single retrieval might return only a partial answer, which is worse than no answer at all. Moreover, if the correct source depends on the specific question, a single index lacks the ability to choose.

Additionally, if a question necessitates a second pass over the corpus after the initial answer, there's no provision for that in a single-index setup.

Enter enterprise agentic retrieval, which addresses these limitations by incorporating a reasoning agent into the loop. The agent determines whether retrieval is necessary, selects the appropriate knowledge base to consult, and may even perform additional retrievals before composing the final answer. This approach is increasingly becoming the standard for enterprise-grade RAG solutions.

Amazon Bedrock Knowledge Bases have evolved to support this need. With the introduction of Managed Knowledge Bases, retrieval is no longer limited to a simple embed-and-fetch operation. Instead, it has transformed into an agentic capability, enabling multi-turn planning, tool-driven retrieval, and the generation of grounded, cited answers.

The AgenticRetrieveStream API provides this functionality as a first-class service feature, rather than requiring users to build it from scratch. However, this evolution introduces a new operational challenge: managing the complex decision-making process of an agent that interacts with multiple knowledge bases and iterates through multiple retrieval steps.

To address these challenges, observability and evaluation must be integrated into the design from the outset, rather than being added as afterthoughts. As of August 2024, AWS CDK (Cloud Development Kit) allows developers to deploy an end-to-end RAG solution using Knowledge Bases for Amazon Bedrock with a single command. This deployment includes a single-shot design, which has undergone significant changes since its initial implementation.

Amazon Bedrock now offers a managed knowledge base (Type: MANAGED), which handles ingestion, storage, indexing, and retrieval processes. The OpenSearch Serverless collection and index, previously required for setup, are no longer necessary as part of the solution. Instead, AgenticRetrieveStream, a component provided by Amazon Bedrock AgentCore, takes over the agentic functions, including multi-turn planning, retrieval, and generating cited answers.

This system is now fully observable and evaluatable, with Amazon CloudWatch dashboards providing comprehensive telemetry spanning seven layers of the system. This post details the construction of such a system, utilizing an agent that can route across multiple knowledge bases, retrieve information through the Gateway, and deliver a grounded, cited answer.

All aspects of the agent's behavior are instrumented and monitored through Amazon CloudWatch. The deployment process is streamlined, with CDK handling the sequencing of stacks and the order of provisioning resources, allowing for a seamless deployment with just one command.