Suppressing Misaligned Behaviors in Language Models, Durably
Undesired model behaviors can emerge or re-emerge at any stage of training. Once observed, how effectively can we suppress them? To study this question, we first construct two model organisms--language models deliberately trained to exhibit a specific misalignment--that differ in what triggers the behavior. The narrow-trigger organism misbehaves only when the prompt contains a specific cue: it…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
