Video games in MRI scanners aim to map brains as they predict and adapt

For decades, neuroscientists have studied the human brain by asking individuals to lie very still inside a brain scanner and perform controlled tasks. While these experiments have provided valuable insights, they often fail to capture the complexity of real life, in which we constantly perceive the world, make predictions, decide on actions, and adjust our behavior based on outcomes.

Now, researchers from Western University, led by neuroscientists Jörn Diedrichsen and Jody Culham, are aiming to study the brain in a more realistic manner by having participants play video games inside an fMRI scanner.

This project, part of the Digital Brain Project, seeks to create a comprehensive database of human brain activity. Supported by Meta, the project brings together researchers from various institutions to explore brain behavior across a wider range of cognitive states than typical brain imaging methods typically cover. Diedrichsen's lab has developed over 60 tasks that can be performed during an fMRI scan, allowing researchers to map brain activity under diverse conditions.

Diedrichsen explains that brain activity can be likened to an extensive landscape, with different experiences and actions moving it to various locations. Conventional resting-state fMRI, while useful for observing spontaneous brain fluctuations, lacks the ability to control where in this landscape the brain goes. Diedrichsen and Culham saw an opportunity to combine their approaches, using video games to study both broad brain activity and the brain's response to interactive environments.

Video games, such as Pac-Man, naturally involve prediction, action, and feedback loops—processes that are distinctly different from tasks like looking at images and pressing buttons. In previous research, Culham's lab found that playing Pac-Man, interacting with the game environment, and receiving immediate feedback creates a "closed loop" between the player and the environment. This loop is missing from traditional brain imaging methods, which generally rely on participants performing isolated tasks.

The Digital Brain Project aims to bridge this gap by incorporating immersive video games into brain imaging studies. These games will feature rich 3D environments and first-person avatars, providing a more engaging and interactive experience for participants. The project will include a core set of tasks, such as kart racing, action games, digital puzzles, and more, while also allowing for the introduction of new tasks and game elements throughout the project.

Western University is upgrading its MRI equipment to support this ambitious study. The institution is investing in a 7-tesla MRI scanner, which offers higher-quality images than conventional systems. Additionally, Western is developing a 32-channel head coil to enhance data collection from deep brain structures like the cerebellum, thalamus, and midbrain. This upgrade is particularly important for Diedrichsen's research, which focuses on the cerebellum and its role in prediction and cognition.

By combining carefully designed cognitive tasks with immersive video games, the Digital Brain Project seeks to explore a broader range of brain states while also capturing the brain's adaptive processes. The upgraded MRI scanner will provide participants with more immersive gaming experiences, allowing researchers to observe how the brain responds to interactive environments in real-time.