Dual-Channel Robust Group-Relative Policy Optimization via Advantage and Sequence-Weight Estimation
Group-relative policy optimization relies on reward-derived advantages and sequence-level likelihood weights, both of which can be sensitive to localized outliers. Extreme rewards can collapse the contrast among clean responses after group normalization, while token-level log-ratio perturbations can alter sequence weights and clipping decisions. We introduce RoVR-GSPO, a dual-channel robust…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
