Audit Where Hacks Pay: Budgeted Strong-Verifier Audits Against Reward Hacking in GRPO
Reinforcement learning with verifiable rewards trains on cheap checks, such as unit tests, that a policy can learn to game, while a stronger check is too expensive to run on every rollout. We study how to spend a limited strong-verifier budget in GRPO, separating which weakly passing rollouts to audit from what reward a caught hack receives. At a fixed policy state with centered advantages, the…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
