[2603.02200] Adaptive Confidence Regularization for Multimodal Failure Detection

Computer Science > Computer Vision and Pattern Recognition arXiv:2603.02200 (cs) [Submitted on 2 Mar 2026] Title:Adaptive Confidence Regularization for Multimodal Failure Detection Authors:Moru Liu, Hao Dong, Olga Fink, Mario Trapp View a PDF of the paper titled Adaptive Confidence Regularization for Multimodal Failure Detection, by Moru Liu and 3 other authors View PDF HTML (experimental) Abstract:The deployment of multimodal models in high-stakes domains, such as self-driving vehicles and medical diagnostics, demands not only strong predictive performance but also reliable mechanisms for detecting failures. In this work, we address the largely unexplored problem of failure detection in multimodal contexts. We propose Adaptive Confidence Regularization (ACR), a novel framework specifically designed to detect multimodal failures. Our approach is driven by a key observation: in most failure cases, the confidence of the multimodal prediction is significantly lower than that of at least one unimodal branch, a phenomenon we term confidence degradation. To mitigate this, we introduce an Adaptive Confidence Loss that penalizes such degradations during training. In addition, we propose Multimodal Feature Swapping, a novel outlier synthesis technique that generates challenging, failure-aware training examples. By training with these synthetic failures, ACR learns to more effectively recognize and reject uncertain predictions, thereby improving overall reliability. Extensive experi...