[2506.13130] ZINA: Multimodal Fine-grained Hallucination Detection and Editing

Computer Science > Computer Vision and Pattern Recognition arXiv:2506.13130 (cs) [Submitted on 16 Jun 2025 (v1), last revised 5 Apr 2026 (this version, v2)] Title:ZINA: Multimodal Fine-grained Hallucination Detection and Editing Authors:Yuiga Wada, Kazuki Matsuda, Komei Sugiura, Graham Neubig View a PDF of the paper titled ZINA: Multimodal Fine-grained Hallucination Detection and Editing, by Yuiga Wada and 3 other authors View PDF HTML (experimental) Abstract:Multimodal Large Language Models (MLLMs) often generate hallucinations, where the output deviates from the visual content. Given that these hallucinations can take diverse forms, detecting hallucinations at a fine-grained level is essential for comprehensive evaluation and analysis. To this end, we propose a novel task of multimodal fine-grained hallucination detection and editing for MLLMs. Moreover, we propose ZINA, a novel method that identifies hallucinated spans at a fine-grained level, classifies their error types into six categories, and suggests appropriate refinements. To train and evaluate models for this task, we construct VisionHall, a dataset comprising 6.9k outputs from twelve MLLMs manually annotated by 211 annotators, and 20k synthetic samples generated using a graph-based method that captures dependencies among error types. We demonstrated that ZINA outperformed existing methods, including GPT-4o and Llama-3.2, in both detection and editing tasks. Comments: Subjects: Computer Vision and Pattern Recogn...