Reinforcement Learning via Brain Feedback for real-time fMRI-based adaptive stimulus generation

Conventional fMRI experiments utilize pre-set task conditions, restricting the ability to investigate brain-stimulus connections freely. To address this limitation, we present Reinforcement Learning via Brain Feedback (RLBF), an innovative framework accompanied by open-source software for adaptive stimulus optimization through real-time functional magnetic resonance imaging (fMRI).

RLBF challenges the usual direction of inference by leveraging neural responses to guide the exploration of stimulus spaces using reinforcement learning, allowing for the optimization of predetermined brain targets, such as regional activity or multivariate neural patterns.

The accompanying Python-based software offers a flexible, modular framework that integrates real-time fMRI data processing, reinforcement learning agents, adaptive stimulus generation, simulation-based testing, and experiment monitoring. Researchers can tailor preprocessing pipelines, reward functions, stimulus spaces, and reinforcement learning strategies to suit various closed-loop neuroimaging applications due to the software's adaptable architecture.

We tested the framework in a proof-of-concept study involving ten participants. Our results showed real-time optimization of a simple visual stimulus space by adjusting checkerboard contrast and frequency to maximize primary visual cortex (V1) activity within a single 10-minute fMRI session. RLBF serves as an extensible foundation for brain-guided stimulus optimization, paving the way for novel approaches to study neural specificity, individualized brain-stimulus relationships, and adaptive experimental design.