[2510.25976] Brain-IT: Image Reconstruction from fMRI via Brain-Interaction Transformer

Computer Science > Computer Vision and Pattern Recognition arXiv:2510.25976 (cs) [Submitted on 29 Oct 2025 (v1), last revised 1 Mar 2026 (this version, v2)] Title:Brain-IT: Image Reconstruction from fMRI via Brain-Interaction Transformer Authors:Roman Beliy, Amit Zalcher, Jonathan Kogman, Navve Wasserman, Michal Irani View a PDF of the paper titled Brain-IT: Image Reconstruction from fMRI via Brain-Interaction Transformer, by Roman Beliy and 4 other authors View PDF HTML (experimental) Abstract:Reconstructing images seen by people from their fMRI brain recordings provides a non-invasive window into the human brain. Despite recent progress enabled by diffusion models, current methods often lack faithfulness to the actual seen images. We present "Brain-IT", a brain-inspired approach that addresses this challenge through a Brain Interaction Transformer (BIT), allowing effective interactions between clusters of functionally-similar brain-voxels. These functional-clusters are shared by all subjects, serving as building blocks for integrating information both within and across brains. All model components are shared by all clusters & subjects, allowing efficient training with a limited amount of data. To guide the image reconstruction, BIT predicts two complementary localized patch-level image features: (i)high-level semantic features which steer the diffusion model toward the correct semantic content of the image; and (ii)low-level structural features which help to initialize t...