KV$^2$: A Self-Refining KV Cache
The memory footprint of the key-value (KV) cache constrains the practical use of long-context models, and it dominates cost when one prefilled context must later serve many different queries. In this reusable setting, query-agnostic compression trades cost against quality: lightweight estimators are cheap but less accurate, whereas full-context reconstruction scoring is more accurate yet…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.