How artificial neural networks build their own learning machinery

Artificial intelligence systems have become increasingly adept at tasks that were once thought to be exclusive to human capabilities. Artificial neural networks, the core component of AI, are capable of recognizing images, processing language, and even generating computer code. These systems adapt their connections as they learn, much like the brain's neural circuits.

Scientists wonder if studying these machine systems could provide insights into how the brain learns. A team of researchers from MIT's McGovern Institute and the K. Lisa Yang Integrative Computational Neuroscience Center published a study in the Proceedings of the National Academy of Sciences (PNAS) that explores this question. They emphasize that a key aspect of biological plausibility for artificial neural networks is their ability to self-organize without direct guidance from a developer.

The researchers demonstrated that artificial neural networks can autonomously build their own learning mechanisms. By following straightforward guidelines regarding the adjustments to be made to connections, a group of randomly linked units can reorganize itself into an efficient information-processing network capable of learning from the data it is trained on.

While current AI technology operates effectively, it may not employ a learning algorithm similar to that used by the brain. Qianli Liao, a postdoctoral fellow at MIT's ICoN Center and lead author of the study, clarifies that engineers design the rules governing how connections change during the learning process. However, Liao and his colleagues caution that AI networks trained in this manner might not provide significant insights into biological learning processes.

The brain does not have an external designer or architect; it must organize itself. Understanding the rules that govern how neural connections are reorganized during learning is crucial and could have implications for developing new therapies and prevention strategies for brain-related disorders. Liao highlights the potential of brain-computer interfaces, which aim to restore communication in individuals with lost language abilities or enable paralyzed patients to control neuroprosthetic limbs.

The success of these interfaces depends on a deep understanding of the rules governing neuron-to-neuron interactions, a task that cannot be accomplished through merely designing a functional circuit. Instead, the value of artificial neural networks to neuroscience should be assessed based on their capacity to self-organize, rather than solely on the connections they form.

To validate their findings, the researchers developed and tested a self-organizing computational network that could learn from data. They began by creating a network with multiple layers of randomly connected neuron-like units. Each unit within a layer follows the same rules for adjusting connections. These units interact with one another based on these rules and develop their own behavior.

The team implemented rules similar to those believed to be responsible for rewiring brain circuits as humans and animals learn from their experiences. When two interconnected neurons fire simultaneously, the connection between them strengthens, following the principle that "neurons that fire together wire together." The team showed that a self-organizing network governed by such rules can learn to recognize patterns and classify images.

These motifs, which emerge through simple rules and later serve as building blocks for functional circuits, reorganize themselves using an optimization strategy akin to the widely used gradient descent algorithm in machine learning. The gradient descent algorithm involves adjusting connections in response to errors to enhance performance.

Unlike most modern AI systems that rely on gradient descent, the team's self-organizing network showed comparable performance. Comparing their self-organizing network to others designed to learn through gradient descent, the researchers found that it performed at a similar level. Poggio suggests that identifying a biological parallel to gradient descent could lead to significant advancements in both AI and neuroscience.

If the synaptic motifs proposed by the researchers are indeed utilized by the brain to perform what gradient descent achieves in artificial neural networks, it would establish a strong foundation of fundamental mechanisms connecting artificial and biological networks. Liao emphasizes that a self-organizing network may not necessarily learn in the same manner as the brain but serves as a crucial starting point for evaluating biological plausibility.

Once self-organization is well-understood as a measure, researchers can then explore further aspects of biological learning. While the primary goal of the study is to enhance our understanding of the brain, Liao believes that this knowledge could also benefit machine learning systems, potentially leading to the development of networks that perform better or in a manner more akin to human learning.