Cortico-subcortical multi-head self-attention as a substrate for cognitive performance
The neocortex is central to mammalian cognition, yet a computational framework that is both biologically constrained and capable of performing complex cognitive tasks remains missing. Here we show that cortico-thalamic circuits are well suited to implement multi-head self- and cross-attention, the mechanism underlying the cognitive abilities of transformer networks. We propose that layer 2/3…
Recent research has unveiled a computational framework rooted in the neocortex that could underpin the complex cognitive abilities of mammals. This framework involves cortico-thalamic circuits capable of executing multi-head self- and cross-attention, a mechanism integral to the performance of transformer networks.
The model hypothesizes that layer 2/3 pyramidal cells in the cortex maintain a recurrent key-value memory, while layer 5 pyramidal cells decode the memory retrieved by a query. The process of generating keys, values, and queries is linked to core and matrix thalamo-cortical projections, spread out across micro- and macro-columns within a cortical region. Essentially, one cortical area operates as an attention head, with the entire cortex functioning as a multi-head self-attention network.
Interestingly, this same thalamo-cortical microcircuit also handles sensory prediction errors, which guide gradient-based synaptic plasticity. Moreover, a reward-prediction error influences the cortical output and re-activation of hippocampal memories via the basal ganglia. The trained network mirrors human intracranial recordings during speech perception.
In summary, this suggested cortico-subcortical attention circuit could potentially serve as a substrate for the cognitive prowess of mammals.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
