$T^5$: Twin-Critic Training for Token-Level Thoughts in Reinforcement Mid-Training
Reinforcement mid-training lets language models learn internal thoughts from unlabeled text, but efficient token-level credit assignment remains challenging. Existing group-relative methods require costly repeated generation. Learned critics offer single-rollout feedback, but accurate return prediction alone does not ensure reliable policy updates. Our analysis shows how training--inference…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
