[2604.01989] Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation

Computer Science > Computer Vision and Pattern Recognition arXiv:2604.01989 (cs) [Submitted on 2 Apr 2026 (v1), last revised 3 Apr 2026 (this version, v2)] Title:Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation Authors:Boyang Gong, Yu Zheng, Fanye Kong, Jie Zhou, Jiwen Lu View a PDF of the paper titled Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation, by Boyang Gong and 4 other authors View PDF HTML (experimental) Abstract:Like a body at rest that stays at rest, we find that visual attention in multimodal large language models (MLLMs) exhibits pronounced inertia, remaining largely static once settled during early decoding steps and failing to support the compositional understanding required for cognitive inference. While existing hallucination mitigation methods mainly target perceptual hallucinations concerning object existence or attributes, they remain inadequate for such cognitive hallucinations that require inter-object relational deduction. Through token-wise attention analysis, we identify this visual inertia as a key factor: attention to semantically critical regions remains persistently focused and fails to dynamically support relational inference. We thereby propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention. Specifically, IVE selects visu...