[2509.21029] FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction

Computer Science > Machine Learning arXiv:2509.21029 (cs) [Submitted on 25 Sep 2025 (v1), last revised 28 Feb 2026 (this version, v3)] Title:FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction Authors:Runqi Lin, Alasdair Paren, Suqin Yuan, Muyang Li, Philip Torr, Adel Bibi, Tongliang Liu View a PDF of the paper titled FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction, by Runqi Lin and 6 other authors View PDF HTML (experimental) Abstract:The integration of new modalities enhances the capabilities of multimodal large language models (MLLMs) but also introduces additional vulnerabilities. In particular, simple visual jailbreaking attacks can manipulate open-source MLLMs more readily than sophisticated textual attacks. However, these underdeveloped attacks exhibit extremely limited cross-model transferability, failing to reliably identify vulnerabilities in closed-source MLLMs. In this work, we analyse the loss landscape of these jailbreaking attacks and find that the generated attacks tend to reside in high-sharpness regions, whose effectiveness is highly sensitive to even minor parameter changes during transfer. To further explain the high-sharpness localisations, we analyse their feature representations in both the intermediate layers and the spectral domain, revealing an improper reliance on narrow layer representations and semantically poor frequency components. Building on this, we propose a Feature Ove...