Where Should a Document Live: Context, Representations, or Parameters?
To answer questions outside of their pre-training data, large language models (LLMs) need access to new information, which can be presented in the context window as documents, encoded into the model's parameters, or injected as latent representations. However, each of these methods comes with different efficiency, cost, and performance trade-offs, with no single winner. We present a controlled…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
