![[2602.18154] FENCE: A Financial and Multimodal Jailbreak Detection Dataset](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.18154] FENCE: A Financial and Multimodal Jailbreak Detection Dataset
Summary
The paper presents FENCE, a bilingual multimodal dataset designed for detecting jailbreaks in financial applications, highlighting vulnerabilities in LLMs and VLMs.
Why It Matters
As AI systems become integral in finance, ensuring their security against jailbreaks is crucial. FENCE addresses a significant gap in resources for training effective detection models, enhancing the safety of AI applications in sensitive domains.
Key Takeaways
- FENCE is a bilingual dataset for training jailbreak detectors in finance.
- The dataset reveals vulnerabilities in both commercial and open-source VLMs.
- A baseline detector trained on FENCE achieves 99% accuracy, indicating its robustness.
- The dataset emphasizes domain realism with finance-relevant queries and image threats.
- FENCE supports safer AI deployment in critical financial applications.
Computer Science > Computation and Language arXiv:2602.18154 (cs) [Submitted on 20 Feb 2026] Title:FENCE: A Financial and Multimodal Jailbreak Detection Dataset Authors:Mirae Kim, Seonghun Jeong, Youngjun Kwak View a PDF of the paper titled FENCE: A Financial and Multimodal Jailbreak Detection Dataset, by Mirae Kim and 2 other authors View PDF HTML (experimental) Abstract:Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vulnerable because they process both text and images, creating broader attack surfaces. However, available resources for jailbreak detection are scarce, particularly in finance. To address this gap, we present FENCE, a bilingual (Korean-English) multimodal dataset for training and evaluating jailbreak detectors in financial applications. FENCE emphasizes domain realism through finance-relevant queries paired with image-grounded threats. Experiments with commercial and open-source VLMs reveal consistent vulnerabilities, with GPT-4o showing measurable attack success rates and open-source models displaying greater exposure. A baseline detector trained on FENCE achieves 99 percent in-distribution accuracy and maintains strong performance on external benchmarks, underscoring the dataset's robustness for training reliable detection models. FENCE provides a focused resource for advancing multimodal jailbreak detection in finance and for supporting safer, more reliable A...
