Design, Optimization, and Deployment of a Hybrid Agentic Retrieval-Augmented Generation (Agentic-RAG) Architecture

Traditional Retrieval-Augmented Generation (RAG) pipelines rely on static vector search for data retrieval, which can lead to vocabulary mismatches and an inability to verify context. This research introduces an Agentic Hybrid RAG architecture that addresses these issues by combining dense semantic search with exact keyword search and employing an autonomous LLM agent.

The system architecture consists of three complementary components: knowledge base ingestion and chunking, dual-engine indexing, and agentic orchestration. The chunking process employs recursive character boundary splitting to maintain semantic continuity while preventing information loss at chunk boundaries. Dense vector search is performed using FAISS IndexFlatIP with L2-normalized embeddings, while sparse lexical search is implemented through the BM25Okapi algorithm.

To overcome the imbalance between the distributions of dense vector inner products and BM25 scores, the system applies feature-level min-max scaling and hybrid fusion with an α value of 0.7, prioritizing semantic context while preserving 30% weight for exact keyword preservation. The challenges encountered during deployment, such as sandbox import errors, CPU compute latency, and tool-calling hallucinations, are addressed by transitioning to a cloud model (Qwen2.5-72B-Instruct) and dynamic score normalization.

The Agentic Hybrid RAG architecture aims to function as a rational, reasoning-capable system that can overcome architectural weaknesses in large language models, providing a blueprint for scaling into knowledge graphs, multi-agent consensus, and automated valuation frameworks.