The transformer as a candidate canonical cortical microcircuit
Neuroscience hypothesizes a canonical microcircuit subserving cortical computation across brain systems. In artificial intelligence, neural networks have become a universal computational substrate. Here, we leverage newly available large-scale data and mapping techniques to compare neural network motifs to the brain. We embedded a spectrum of candidate microcircuit architectures within a common…
Neuroscience suggests the existence of a canonical microcircuit that underpins cortical computation across various brain systems. In the field of artificial intelligence, neural networks have emerged as a universal computational substrate. Researchers have now utilized large-scale data and advanced mapping techniques to compare neural network motifs with the brain.
To achieve this, they embedded a range of candidate microcircuit architectures within a common spatial macrocircuit, and measured their alignment with a comprehensive set of cortical signatures.
The findings reveal that a particular circuit type, the transformer, stands out in its ability to capture fine-grained unit-level circuit details, population-level functional maps, and neuronal activity patterns. Additionally, the transformer exhibits systems-level wiring efficiency and aligns with behavioral demands. The critical factor enabling the transformer's success is a learned multiplicative component that makes relational information available without the need for extensive long-range wiring.
These results indicate a striking convergence between computational motifs found in natural neural circuits and those employed in artificial intelligence.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
