ARM: Attention with Routed-Memory for Learnable Sparse Control
Despite advances in long-context inference, large language models (LLMs) remain fundamentally limited by the key-value (KV) caching mechanisms that are necessary for stable computation. Techniques such as selective token eviction and pruning have vastly mitigated these issues, but often discard core information to manage the growing cache. In this paper, we propose Attention with Routed Memory…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
