![[2602.13289] Evaluating the Impact of Post-Training Quantization on Reliable VQA with Multimodal LLMs](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13289] Evaluating the Impact of Post-Training Quantization on Reliable VQA with Multimodal LLMs
Summary
This paper evaluates the effects of Post-Training Quantization (PTQ) on the reliability and accuracy of Visual Question Answering (VQA) using Multimodal Large Language Models (MLLMs). It identifies how quantization impacts model performance and proposes methods to mitigate rel...
Why It Matters
As MLLMs become integral in applications requiring both efficiency and reliability, understanding the implications of quantization is crucial. This study provides insights into optimizing model performance for deployment on edge devices, which is vital for real-world applications in AI.
Key Takeaways
- Post-Training Quantization (PTQ) negatively impacts both accuracy and reliability in VQA tasks.
- Data-aware quantization methods can mitigate some reliability issues caused by quantization.
- The Selector confidence estimator improves reliability in quantized multimodal settings.
- Combining int4 MBQ with the Selector achieves a favorable efficiency-reliability trade-off.
- This study is the first to systematically link quantization effects with reliability in multimodal AI applications.
Computer Science > Computer Vision and Pattern Recognition arXiv:2602.13289 (cs) [Submitted on 8 Feb 2026] Title:Evaluating the Impact of Post-Training Quantization on Reliable VQA with Multimodal LLMs Authors:Paul Jonas Kurz, Tobias Jan Wieczorek, Mohamed A. Abdelsalam, Rahaf Aljundi, Marcus Rohrbach View a PDF of the paper titled Evaluating the Impact of Post-Training Quantization on Reliable VQA with Multimodal LLMs, by Paul Jonas Kurz and 4 other authors View PDF HTML (experimental) Abstract:Multimodal Large Language Models (MLLM) are increasingly deployed in domains where both reliability and efficiency are critical. However, current models remain overconfident, producing highly certain but incorrect answers. At the same time, their large size limits deployment on edge devices, necessitating compression. We study the intersection of these two challenges by analyzing how Post-Training Quantization (PTQ) compression affects both accuracy and reliability in Visual Question Answering (VQA). We evaluate two MLLMs, Qwen2-VL-7B and Idefics3-8B, quantized with data-free (HQQ) and data-aware (MBQ) methods across multiple bit widths. To counteract the reduction in reliability caused by quantization, we adapt the Selector confidence estimator for quantized multimodal settings and test its robustness across various quantization levels and out-of-distribution (OOD) scenarios. We find that PTQ degrades both accuracy and reliability. Data-aware methods soften the effect thereof. The...
