Citations That Survive a TypeScript Refactor

The code snippet provided demonstrates a common approach to implementing citations in Retrieval-Augmented Generation (RAG) applications. The developer numbers the retrieved chunks, asks the language model to cite by number, and then renders the numbers as clickable links. This initial implementation is straightforward and works immediately, making it suitable for production use.

However, the author highlights a significant issue with this approach. Even a minor change upstream, such as altering the chunk size or adding a reranker, can cause citations to break silently. This happens because the citations are based on the position of the chunks in the array, which changes with any modification to the retrieval process. Consequently, a citation that previously pointed to the correct chunk now points to a different chunk, leading to inaccurate information being presented to the user.

To address this problem, the author proposes a solution involving a more robust citation system. They introduce a `SourceRef` type that includes several key fields: `docId`, `contentHash`, `start`, `end`, and `revision`. Each of these fields serves a specific purpose:

- `docId` remains stable throughout the process of re-chunking, re-embedding, and re-ranking.

- `contentHash` uses a SHA256 hash of the normalized chunk text, allowing for accurate identification of the exact text even if its position in the chunk array changes.

- `start` and `end` provide character offsets within the document, enabling precise location and highlighting of the cited text, even if the chunking changes.

- `revision` records the specific version of the document that was retrieved, ensuring that citations reference the correct content even if the document has been modified since retrieval.

The author emphasizes that these fields collectively make citations degrade gracefully rather than vanish when upstream changes occur. By using `contentHash` and location information, the system can still locate and render the correct information, even if the underlying chunking has changed.

Additionally, the author introduces a type `Cited<T>` that enforces the rule that every factual sentence must carry a source. This type ensures that any claim or assertion in the generated text is linked to a valid `SourceRef`. This prevents the accidental creation of a claim without a citation and adds a layer of type safety to the system.

The code also includes a mechanism for handling the model's responses. Since the model cannot directly emit a `contentHash`, the author suggests providing it with short, opaque labels instead. These labels are mapped to their corresponding `ref` values (which include the other citation fields) in a Map. The model then generates output using these labels, which can be resolved back to the actual citation data after the fact.

This approach ensures that the citations can be accurately reconstructed and maintained throughout the entire RAG process, from retrieval to generation and storage. By anchoring citations to stable identifiers and carefully separating factual content from citation information, the system becomes more resilient to changes and provides users with reliable and verifiable information.