Phrase-Level Speculative Decoding on a Frozen Target
Speculative decoders raise throughput by banking more target tokens per forward pass, and Medusa, EAGLE-3, DFlash and DSpark all do it the same way: more candidates that survive verification. All of them draft in the target's own vocabulary, so one node of the candidate tree is one target token, and the tokenizer makes that unit look like a constant. It is not. BPE's pretokenizer splits on…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
