Beyond Reward Scoring: Online Multi-Agent Preference Learning for Multimodal Response Generation
Multimodal preference alignment often follows a score-based RLHF pipeline: a reward model is first trained to learn human preferences, then used to score on-policy responses and optimize the response model's generation. This turns preference supervision into a scalar signal for generation: a model may learn to produce answers with higher preference scores, but is rarely trained explicitly to…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
