GIST: Gibbs-Targeted Hierarchical Data Selection for Offline RLVR
Offline reinforcement learning with verifiable rewards (RLVR) trains on a fixed pool of sampled responses and typically supervises every token of every response, although responses and tokens are not equally informative. We study which ones to supervise through the Gibbs-optimal policy of KL-regularized RLVR, which offline training targets, and measure the mismatch between a policy and this…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
