Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime
Computer programmers chain together the same reusable computational primitives, for example, conditional statements or loops, but in different orders to implement different algorithms. In contrast, existing language model architectures always execute their layers in the same order. We introduce Library of Layers (LoL), an architecture that dynamically orders its layers during inference to solve…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
