[2603.29405] Hallucination-aware intermediate representation edit in large vision-language models

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.29405 (cs) [Submitted on 31 Mar 2026] Title:Hallucination-aware intermediate representation edit in large vision-language models Authors:Wei Suo, Hanzu Zhang, Lijun Zhang, Ji Ma, Peng Wang, Yanning Zhang View a PDF of the paper titled Hallucination-aware intermediate representation edit in large vision-language models, by Wei Suo and 5 other authors View PDF HTML (experimental) Abstract:Large Vision-Language Models have demonstrated exceptional performance in multimodal reasoning and complex scene understanding. However, these models still face significant hallucination issues, where outputs contradict visual facts. Recent research on hallucination mitigation has focused on retraining methods and Contrastive Decoding (CD) methods. While both methods perform well, retraining methods require substantial training resources, and CD methods introduce dual inference overhead. These factors hinder their practical applicability. To address the above issue, we propose a framework for dynamically detecting hallucination representations and performing hallucination-eliminating edits on these representations. With minimal additional computational cost, we achieve state-of-the-art performance on existing benchmarks. Extensive experiments demonstrate the effectiveness of our approach, highlighting its efficient and robust hallucination elimination capability and its powerful controllability over hallucinations. Code i...