RAG as an MCP Server: How I Let Claude Code Search My Own Documents

This article aims to assist AI developers who wish for Claude Code to retrieve relevant passages from their personal documents using a small MCP server. Claude Code can currently read individual files from a project, but when dealing with numerous lengthy PDFs like reports or manuals, this process consumes excessive tokens and forces Claude to guess which file to open first.

A more efficient solution involves conducting a targeted search across all documents, returning only the matching passages. The author had previously implemented such a search in their RAG (Retrieval-Augmented Generation) chatbot and built the MCP server primarily to understand the practical workings of MCP. Surprisingly, the adaptation process proved to be minimal.

Below, the author demonstrates how to integrate the search functionality from their RAG pipeline into Claude Code using a compact MCP server, along with instructions on how to connect and test it.

MCP, or Model Context Protocol, is an open protocol that enables AI agents to utilize tools from a server. The server defines its tools, and the client, such as Claude Code, determines when to invoke them. An example of this concept can be seen in a Gmail MCP server, which allows Claude to search through your emails or generate drafts within your Gmail account.

Compared to a REST API, employing MCP offers the advantage of building the server once, allowing any MCP client, including Claude Code, Claude Desktop, or Cursor, to utilize it seamlessly. A REST API would necessitate writing a unique integration for each client.

The MCP server employed in this project employs Streamable HTTP as the transport protocol rather than stdio. While stdio requires each client to initiate its own copy of the server, HTTP facilitates a shared server that multiple clients can concurrently connect to. During the design phase, four key decisions were made. Firstly, the server refrains from invoking any LLM (Language Model) and solely returns the pertinent passages it discovers.

The client, already possessing a model, dictates which model will answer the query. This approach eliminates the necessity for API keys and renders the server entirely cost-free per request. Secondly, the server maintains its own knowledge base. In the author's chatbot, users upload files for each chat session, and the MCP server is designed to remain continuously available.

Consequently, the server reads the folder of documents once, stores them on disk using Chroma, and subsequently indexes them. Two tools were deemed sufficient for the client's needs: one for searching the documents and another for listing the files in the knowledge base. These two functionalities encompass all essential requirements, preventing any unnecessary additions.

The search functionality mirrors that of the chatbot, with the search_vectorstore function serving as a common component for both the agent and the MCP server. This ensures consistent results and simplifies testing to a single implementation. The ingestion process begins by loading a folder containing PDF, TXT, and Markdown files.

Each file is subsequently split into chunks approximately 500 characters long, which are then stored in the knowledge base. The metadata associated with each page is tagged with its file name, enabling the server to associate each piece of information with its source document. After the ingestion process, the server deletes the existing knowledge base using shutil.rmtree and reconstructs it from the latest set of documents.

This approach guarantees that any files removed from the folder are consequently excluded from the search results. Moreover, the embeddings are generated locally using BAAI/bge-base-en-v1.5 via Hugging Face, negating the need for any API keys during the ingestion phase.