Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR
Reinforcement learning with verifiable rewards (RLVR) and on-policy distillation (OPD) have emerged as two dominant methods for post-training reasoning LLMs. Prior work uses OPD's dense token-level supervision to complement the sparse RL reward, fusing the two signals within a single step: either as a \emph{weighted-additive combination} or a \emph{teacher-modulated rescaling} of the RL…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
