How visual learning happens in the brain

Neural pathways in the brain constantly change as we learn and interact with our surroundings. Scientists at MIT’s McGovern Institute and York University are studying this brain plasticity using detailed brain activity analysis and computational modeling. The researchers trained an artificial neural network with brain-like architecture to identify objects, similar to animals being trained.

As the model's performance improved, its internal structure reorganized in ways that mirrored changes detected in the animal brains. The work, published in Nature Communications, shows how visual processing changes support animals' ability to learn to distinguish new objects. Modeling these changes could help predict how training reshapes perception and inform educational strategies for various learners.

Visual-processing areas in the brain work together to interpret eye input, then communicate with other brain regions to give visual information meaning and guide behavior. The team focused on the inferior temporal (IT) cortex, a crucial part of the brain's visual object-processing network. When visual information reaches the IT cortex, it has clearly represented key object features, allowing researchers to decode object identity and predict identification errors based on neural activity patterns.

The team recorded IT cortex activity from animals as they viewed and identified images of objects. Some animals were untrained, while others had learned to identify similar objects, even when presented at different sizes, angles, or backgrounds. The overall pattern of activity in the IT cortex was similar in trained and untrained animals, indicating that learning had not radically altered this high-level visual representation.

However, subtle differences in how IT cortex neurons responded to images were found in trained animals compared to untrained ones. The researchers created artificial neural networks, mapping their internal components to the IT cortex, to investigate how these minor changes may contribute to learning. Some animal models showed learning behavior akin to the trained animals.

In those models, the IT-like stage underwent changes resembling the learning-related alterations observed in the IT cortex of trained animals. While gradient descent, commonly used to train artificial intelligence, is generally considered biologically implausible as a direct model of brain learning, the strong match in learning effects between animals and the model suggests artificial neural networks can provide insights into biological learning at a useful level of abstraction.

The study demonstrates that in silico versions of future experiments can be built and used to ask "what if" questions, potentially predicting new findings beyond experimental intuition. Most learning-related changes in the model occurred outside of the IT cortex, indicating that multiple areas need to change during the learning process.

The team's model will aid researchers in understanding how downstream brain areas contribute to learning. The researchers emphasize that their study provided more granular brain activity measurements than would be possible in humans and that the animal brains are organized similarly to ours, making their experiments directly relevant to human learning.

Understanding the impact of plasticity in the IT cortex could help researchers design new learning strategies for humans. MIT Professor James DiCarlo, who is also the director of the MIT Quest for Intelligence, explains that their study supports the concept that learning new objects involves changes in synaptic connections downstream of the visual system, preserving the visual system's integrity.