[2604.04488] A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models

Computer Science > Computer Vision and Pattern Recognition arXiv:2604.04488 (cs) [Submitted on 6 Apr 2026] Title:A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models Authors:Tianmeng Fang, Yong Wang, Zetai Kong, Zengzhen Su, Jun Wang, Chengjin Yu, Wei Wang View a PDF of the paper titled A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models, by Tianmeng Fang and 6 other authors View PDF HTML (experimental) Abstract:Multimodal large language models have become an important infrastructure for unified processing of visual and linguistic tasks. However, such models are highly susceptible to backdoor implantation during supervised fine-tuning and will steadily output the attacker's predefined harmful responses once a specific trigger pattern is activated. The core challenge of backdoor defense lies in suppressing attack success under low poisoning ratios while preserving the model's normal generation ability. These two objectives are inherently conflicting. Strong suppression often degrades benign performance, whereas weak regularization fails to mitigate backdoor behaviors. To this end, we propose a unified defense framework based on patch augmentation and cross-view regularity, which simultaneously constrains the model's anomalous behaviors in response to triggered patterns from both the feature representation and output distribution levels. Specifically, patch-level data augme...