ReplaySSM: Cache SSM Inputs, Not State
State-space models (SSMs) maintain a fixed-size recurrent state, but reading and writing it at every decoding step can bottleneck hybrid models. This eager state management causes large state I/O with low arithmetic intensity, leaves speculative decoding no cheap rollback, and feeds state quantization error into subsequent recurrent updates. We introduce ReplaySSM, which unrolls the recurrence to…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
