LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models
Looped Transformers improve parameter efficiency by repeatedly applying the same Transformer layers. However, generating each token still involves sequential computation and repeated access to model weights, leading to high decoding latency. Recent pipelined self-speculative decoding methods reduce this delay by drafting tokens from early-loop predictions and asynchronously advancing tokens at…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
