Your RAG System Can Be Healthy and Still Be Wrong

Artificial intelligence systems can provide incorrect answers, even if they appear to function correctly from a technical standpoint. This issue arises when the information used to generate an answer is outdated or no longer trustworthy. Consider a policy assistant in an enterprise setting. When a policy is updated, the retrieval index may still contain the previous version.

If an employee asks a question about the new policy, the system may still provide an answer based on the outdated information. Although the system processes the request successfully, the answer may be inaccurate.

The problem lies not solely in the language model's generation capabilities but in the application's ability to verify the accuracy and relevance of the information it uses. A successful request does not guarantee a correct answer. The retrieval-augmented generation (RAG) pipeline consists of several components, each of which can work perfectly while the overall system fails.

For instance, vector search might retrieve documents without checking their relevance or freshness, while the language model may summarize the retrieved text accurately but still produce an answer that contradicts the current policy.

To address this issue, it is crucial to establish a way to determine whether the evidence used in the system is still valid. One approach is to ensure that the application does not treat a retrieved document as authoritative when it cannot confirm that the indexed version matches the current source version. Consider a scenario where the authoritative document store and the vector index maintain separate representations of a policy.

The authoritative source contains the current document version, while the index contains the version available when the document was last processed.

In this example, the document ID 'policy-42' has a version 'v8' in the authoritative source, but 'v7' in the retrieved index. If the application does not explicitly check the version freshness, it may mistakenly use the older, outdated content. To mitigate this data-consistency problem, it is essential to record the version of each retrieved document alongside the version expected by the authoritative source. By doing so, the application can make an informed decision about whether the evidence can be used.

Implementing a document-freshness gate can help enforce this assurance. In Python, a function can be created to validate the freshness of retrieved documents. This function takes the candidate documents and a function to obtain the current version of each document from a trusted source of truth. It then separates the documents into two groups: accepted (where the indexed version matches the current version) and rejected (where the indexed version is outdated). The function returns a tuple containing the accepted and rejected documents.

By implementing this freshness check, the application gains an observable basis for deciding whether the evidence behind its answer is current, relevant, and trustworthy. However, it is important to note that the reliability of this check depends on the source of truth and the consistency guarantees around it. If the source of truth reads from a stale cache or cannot account for in-progress updates, the check may provide a false sense of safety.

Therefore, the source of truth and its consistency guarantees should be considered part of the system's design and not be overlooked.