SetGRPO: Calibrating Grader-Induced Policy Optimization
Rubric-based reinforcement learning uses fine-grained grader feedback to train large language models on open-ended tasks, but grader randomness can alter GRPO updates in ways that score discrepancies alone cannot characterize. Because GRPO jointly normalizes rewards within each response group and combines the resulting training coefficients with response-level gradients, a scoring discrepancy can…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
