Stochastic Pruning of the DPO Computation Graph: 1.4x Faster Training and Lower Peak Memory
Direct preference optimization (DPO) incurs training costs in output scoring, sequence execution, and backpropagation, so reducing supervision alone does not necessarily save computation. We introduce a unified framework for stochastic pruning of the DPO computation graph that independently samples response positions for scoring, response prefixes for execution, and preference branches for…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
