LatComm: Reusable Context Compression for Latent Communication between Large Language Models
Systems that combine several large language models (LLMs) often use one context for many questions. When the context is passed between models as text, the receiving model processes all of its tokens again for every question. Avoiding this cost requires a context representation that is compact, reusable across questions, and readable by a different LLM. Existing compression and latent…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.