![[2602.21819] SemVideo: Reconstructs What You Watch from Brain Activity via Hierarchical Semantic Guidance](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.21819] SemVideo: Reconstructs What You Watch from Brain Activity via Hierarchical Semantic Guidance
Summary
The paper presents SemVideo, a novel framework that reconstructs videos from brain activity using hierarchical semantic guidance, addressing key challenges in fMRI-to-video reconstruction.
Why It Matters
Understanding how the brain processes visual information is crucial for advancements in neuroscience and AI. SemVideo's approach could enhance our ability to decode and interpret brain signals, with implications for both medical research and AI development.
Key Takeaways
- SemVideo improves fMRI-to-video reconstruction by addressing visual representation inconsistencies and temporal coherence issues.
- The framework utilizes hierarchical semantic cues to enhance alignment and motion adaptation during video reconstruction.
- Experiments show SemVideo sets a new state-of-the-art in performance on established datasets.
Computer Science > Computer Vision and Pattern Recognition arXiv:2602.21819 (cs) [Submitted on 25 Feb 2026] Title:SemVideo: Reconstructs What You Watch from Brain Activity via Hierarchical Semantic Guidance Authors:Minghan Yang, Lan Yang, Ke Li, Honggang Zhang, Kaiyue Pang, Yizhe Song View a PDF of the paper titled SemVideo: Reconstructs What You Watch from Brain Activity via Hierarchical Semantic Guidance, by Minghan Yang and 5 other authors View PDF HTML (experimental) Abstract:Reconstructing dynamic visual experiences from brain activity provides a compelling avenue for exploring the neural mechanisms of human visual perception. While recent progress in fMRI-based image reconstruction has been notable, extending this success to video reconstruction remains a significant challenge. Current fMRI-to-video reconstruction approaches consistently encounter two major shortcomings: (i) inconsistent visual representations of salient objects across frames, leading to appearance mismatches; (ii) poor temporal coherence, resulting in motion misalignment or abrupt frame transitions. To address these limitations, we introduce SemVideo, a novel fMRI-to-video reconstruction framework guided by hierarchical semantic information. At the core of SemVideo is SemMiner, a hierarchical guidance module that constructs three levels of semantic cues from the original video stimulus: static anchor descriptions, motion-oriented narratives, and holistic summaries. Leveraging this semantic guidance,...