Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learning
Subliminal Learning (SL) is a recently identified phenomenon in which a student model acquires downstream task capabilities by matching seemingly unrelated auxiliary outputs from a teacher, despite never observing task labels, task-specific outputs, or the original training data. While recent studies have identified where subliminal signals may reside, the optimization mechanism underlying this…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
