The Transformer Revolution, Part 1: Dynamic Processing through Output- Weight Interconnections
This paper offers a new interpretation of the Transformer during inference. Against the "stochastic parrot" view that large language models merely reproduce statistical regularities learned in training, we argue that Transformers construct and apply prompt-dependent transformations whose parameters are generated during inference. We call this form of computation SIDPP: Sequence-level Interactive…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
