[2604.09222] GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking

Computer Science > Sound arXiv:2604.09222 (cs) [Submitted on 10 Apr 2026] Title:GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking Authors:Yunqiang Wang, Hengyuan Na, Di Wu, Miao Hu, Guocong Quan View a PDF of the paper titled GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking, by Yunqiang Wang and 4 other authors View PDF HTML (experimental) Abstract:Audio large language models (ALLMs) enable rich speech-text interaction, but they also introduce jailbreak vulnerabilities in the audio modality. Existing audio jailbreak methods mainly optimize jailbreak success while overlooking utility preservation, as reflected in transcription quality and question answering performance. In practice, stronger attacks often come at the cost of degraded utility. To study this trade-off, we revisit existing attacks by varying their perturbation coverage in the frequency domain, from partial-band to full-band, and find that broader frequency coverage does not necessarily improve jailbreak performance, while utility consistently deteriorates. This suggests that concentrating perturbation on a subset of bands can yield a better attack-utility trade-off than indiscriminate full-band coverage. Based on this insight, we propose GRM, a utility-aware frequency-selective jailbreak framework. It ranks Mel bands by their attack contribution relative to utility sensitivity, perturbs only a selected subset of bands, and learns a reusable universal per...