Behavioral and brain responses to language reflect different levels of linguistic representation

The processing of human language can be examined through both behavioral and brain activity. However, it is uncertain if these two types of data reflect the same information. One prevailing theory suggests that both behavioral and neural responses are primarily influenced by the effort required for processing. This effort is often gauged through word surprisal, considering factors such as word frequency and length.

Concurrently, neural responses have demonstrated the ability to encode more intricate aspects of linguistic content, including meaning. To explore these contrasting perspectives, researchers utilized neural network language models to quantify these options. They aimed to compare, within the same analytical computational framework, the predictive capacity of low-dimensional effort-based predictors and high-dimensional embedding representations that encapsulate contextualized linguistic content, including meaning.

Analyzing eight behavioral datasets and five neural datasets (comprising four fMRI and one ERP), the study found that processing effort accounts for a significant proportion of variance in both behavioral and neural measures of language processing. This aligns with previous findings. Nevertheless, for brain responses—notably, not for behavioral measures)—embedding representations revealed substantial predictive power beyond estimates of processing effort.

Consequently, these findings imply that neural data offer access to the intricate, high-dimensional dynamics of language comprehension, whereas behavioral data reveal a reduction of these dynamics to a limited set of theoretically grounded properties of contextualized linguistic input.