Minimally Invasive Steering of Language Models
Pre-logit steering adapts a frozen language model to a test-time reward by adding vectors to its final hidden states. Unregularized reward optimization can substantially alter the output distribution and degrade generation quality. We propose Minimally Invasive Steering Vector Optimization (MISVO), which penalizes interventions using the local KL geometry of the induced token distribution. The…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
