When Less is More: An Active Learning Approach to Optimizing Stimuli Selection in fMRI-to-Image Reconstruction

Abstract

Reconstructing visual experiences from observed fMRI brain activity is a fundamental problem in computational neuroscience, though practical applications are limited by the high cost and time demands of data collection. This thesis investigates whether active learning can improve sample efficiency in fMRI-to-image decoding by strategically selecting which stimuli to present during scanning sessions, reducing the amount of per-subject data required to learn brain-to-embedding mappings for accurate image reconstruction. We evaluate a covariance-based active learning strategy for mapping voxel activity to CLIP image embeddings, benchmarking its performance against random sampling across data settings ranging from fully synthetic to fully empirical. Our findings show that in well-specified linear settings, active learning consistently outperforms random sampling, yielding the most pronounced gains near or above the interpolation threshold. However, in the fully empirical setting with real fMRI features and real CLIP responses, the gap is eliminated entirely as performance substantially degrades across both strategies. We attribute this breakdown to model misspecification: when the model fails to appropriately approximate the true underlying mapping between voxels and embeddings, uncertainty estimates become uninformative and active learning offers minimal advantage over random sampling. We conclude that the efficacy of active learning in neural decoding depends critically on model specification, and propose an extension into non-linear decoding frameworks to recover the theoretical gains of active learning for real-world stimulus selection.