AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs
Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
