Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control
In reinforcement learning with verifiable rewards (RLVR), imperfect verifiers can reward incorrect responses, creating opportunities for reward hacking. Using gradient flow with a fixed verifier, we characterize the conditions under which reward rises while correctness falls. We then show that the observations available during RLVR are, in general, insufficient to detect or identify accepted…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
