Larger language models better align with neural representations of natural language

A recent study explored the relationship between the size of large language models (LLMs) and their capacity to mirror the neural representations of natural language processing in the human brain. Transformer-based LLMs, which have grown in complexity over time, were examined to understand this correlation. By employing a subset of these models that were trained on a constant dataset, researchers were able to distinguish the impact of model size from other factors like architecture and training set size.

The experimental design involved using electrocorticography (ECoG) to capture neural activity in epilepsy patients while they listened to a 30-minute naturalistic audio narrative. Researchers then utilized contextual embeddings from each hidden layer of the LLMs to predict word-level neural signals. The findings confirmed that larger LLMs were more adept at capturing the structural nuances of natural language and delivering accurate predictions of neural activity. Previous research had established a similar trend, and these findings reinforced it.

In addition to finding that larger models performed better, researchers also discovered a logarithmic relationship between model size and encoding performance. The optimal layer for encoding appeared to peak in earlier layers as the model size increased. Moreover, variations in the most effective layer were observed across different brain regions, suggesting an organized hierarchy for language processing within the human brain.