Urgent.News

What's breaking now, across hundreds of outlets.

AI

Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

Speculative decoding accelerates large language model inference by using a low-cost draft model to propose tokens that the full-size target model verifies in parallel. Parallel and semi-autoregressive (semi- AR) drafters improve drafting efficiency by proposing an entire block in a single forward pass, but training them raises a new difficulty: the draft distribution for a given position depends…

We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.

Read the original at arxiv.org →

Report a problem with this story

More in AI

Why the AI guys built tech that terrifies them

AI leaders like Dario Amodei want to win the AI race — and they also want to slow the race down. That makes sense, says journalist Kevin Roose.

  • AI leaders fear building powerful AI systems.
  • 50% chance AI could cause doomsday.
  • Proposed UN-like standards body for AI regulation.

More from Wednesday 7 October →