Task Context Shapes Component Roles in Language Models
Large language models reuse the same attention heads across tasks, but identifying these shared heads does not explain what they do differently in each task. A natural source of inspiration is task-dependent interactions among shared brain regions, which motivate a network view of language-model computation. Here, we compare fixed model components across tasks using identical graph facts,…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
