Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection
Retrieval-augmented generation (RAG) imposes a prefill cost proportional to retrieved context length, and -- with Transformer backbones -- a KV-cache that grows with each generated token. State-Space Models (SSMs) avoid the second cost by construction; we eliminate the first, collapsing prefill from $O(L_{context})$ to $O(1)$ per query. We introduce PRECOG (Pre-Computed Context Injection), a…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
