5 RAG Mistakes That Leak Private Docs Into Chat Answers

This article outlines five common mistakes that can lead to confidential data being accidentally exposed through Retrieval-Augmented Generation (RAG) systems used in chatbots. The key issues include:

1. Not securing documents with proper audience information. Each chunk of data should have a tenant and audience attribute that determines who may access it. Chunks need to be indexed so the system can filter based on these permissions.

2. Filtering results after retrieving them rather than filtering before. Ranking results should incorporate audience permissions so that only relevant chunks are returned to the LLM. This prevents the LLM from seeing private information unnecessarily.

3. Not setting a high enough score threshold for LLM input. If too many irrelevant chunks are retrieved, the model may hallucinate based on weak or unrelated evidence. Setting a high similarity score filter helps ensure only relevant, trustworthy inputs are used.

4. Treating retrieved text as authoritative instructions. Models should be told the retrieved content is data, not instructions, and should cite the sources they use. Fencing off the content and requiring citations helps prevent injection attacks.

5. Not validating that the model cites its sources properly before returning an answer. Models sometimes make up source IDs or omit citations altogether. The system should validate citations against the original documents before the response is sent back to the user.

The article presents a checklist to secure RAG systems, including adding tenant and audience information to each chunk, filtering both before and after search results are ranked, using a high score threshold, treating retrieved text as data not instructions, and validating citations. By following these guidelines, developers can prevent accidental data leaks in chatbot applications.