Agentic RAG: making retrieval a decision the agent controls

Agentic RAG is a refinement of the Retrieval-Augmented Generation (RAG) approach, where the decision-making process of retrieval is controlled by the agent, rather than being a fixed pipeline step. In classic RAG, retrieval occurs once before generation, with the same pipeline run every time. Agentic RAG, however, introduces a decision loop, where the agent determines when to retrieve, what query to use, and whether the results are sufficient.

The key difference between classic RAG and agentic RAG lies in who is in charge of retrieval. In classic RAG, retrieval is a pre-step bolted onto the prompt, while in agentic RAG, retrieval becomes a tool that the agent can call, evaluate, and re-call as needed. This shift allows agentic RAG to treat retrieval as just one of its tools, rather than a fixed part of the pipeline.

Agentic RAG encompasses three capabilities that differentiate it from classic RAG:

1. Retrieval on demand: The agent decides whether retrieval is necessary for a given question. For instance, a simple greeting or a follow-up question that can be answered from context may not require a search.

2. Query formulation: The user's question may not be an ideal search query. Agentic RAG allows the agent to rewrite the query, expanding acronyms, splitting complex questions into sub-questions, or rephrasing them according to the corpus's format.

3. Result grading and iteration: The agent inspects the retrieved results and evaluates their relevance and sufficiency. If the results are inadequate, the agent can reformulate the query and search again, or switch to a different source. This is similar to techniques like Self-RAG, where the generator critiques its own retrieval, or Corrective RAG, which scores the retrieved set and triggers a fallback if needed.

Agentic RAG does not replace the classic RAG architecture; instead, it wraps a decision-maker around the retrieval step, allowing for greater flexibility and robustness. The decision to retrieve, query formulation, and result evaluation are all part of the agent's action layer, rather than being separate from the rest of the system. This integration allows agentic RAG to offer better recall and robustness for multi-step questions or mixed sources, at the cost of additional model turns.

In summary, agentic RAG represents a significant evolution of the RAG approach, giving the agent full control over retrieval, query formulation, and result evaluation. This control enables agentic RAG to handle more complex and varied questions, improving both the efficiency and effectiveness of the system.