CVPR Findings2026
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Yu Yan, Sheng Sun, Shengjia Cheng, Teli Liu, Mingfeng Li, Min Liu
Abstract
Vision-Language Models (VLMs) with multimodal reasoning capabilities are high-value attack targets, given their potential for handling complex multimodal harmful tasks. Mainstream black-box jailbreak attacks on VLMs work by distributing malicious clues across modalities to disperse model attention and bypass safety alignment mechanisms. However, these adversarial attacks rely on simple and fixed image-text combinations that lack scalable attack complexity, limiting their effectiveness for red-teaming VLMs' continuously evolving reasoning capabilities. We propose COMET (CrOss-Modal Entanglement AtTack), which is a scalable approach that extends and entangles information clues across modalities to exceed VLMs' trained and generalized safety alignment patterns for jailbreak. Specifically, knowledge-scalable reframing extends harmful tasks into multi-hop chain instructions, cross-modal clue entangling migrates visualizable entities into images to build multi-modal reasoning links, and cross-modal scenario nesting uses multimodal contextual instructions to steer VLMs toward detailed harmful outputs. Experiments across multiple advanced VLMs show COMET achieves over 94% attack success rate, outperforming the best baseline by 29%.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Jailbreak Large Vision-Language Models Through Multi-Modal Linkage cites Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
- Jailbreak Large Vision-Language Models Through Multi-Modal Linkage cites FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts cites Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
Builds on13
- GAIA: a benchmark for General AI AssistantsGrégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun et al.ICLR 2024 · 716 citations
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual PromptsYichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang et al.AAAI 2025 · 350 citations
- Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language ModelsErfan Shayegani, Yue Dong, Nael B. Abu-GhazalehICLR 2024 · 271 citations
- Attacking Vision-Language Computer Agents via Pop-upsYanzhe Zhang, Tao Yu, Diyi YangACL 2025 · 99 citations
- Jailbreak Large Vision-Language Models Through Multi-Modal LinkageYu Wang, Xiaofei Zhou, Yichen Wang, Geyuan Zhang et al.ACL 2025 · 51 citations
Related papers
- TVChain: Leveraging Textual-Visual Prompt Chains for Jailbreaking Large Vision-Language ModelsHao Yu, Ke Liang, Junxian Duan, Jun Wang et al.AAAI 2026
- VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential AttackShiji Zhao, Shukun Xiong, Yao Huang, Jin Yan, Zhenyu Wu, Jiyang Guan, Ranjie Duan, Jialing Tao, Hui Xue, Xingxing WeiCVPR Findings 2026
- JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language ModelsJiaxin Song, Yixu Wang, Jie Li, Xuan Tong et al.NeurIPS 2025 · 14 citations
- Toward Universal and Transferable Jailbreak Attacks on Vision-Language ModelsKaiyuan Cui, Yige Li, Yutao Wu, Xingjun Ma et al.ICLR 2026 · 4 citations
- Jailbreaking Vision-Language Models Through the Visual ModalityAharon Azulay, Jan Dubiński, Zhuoyun Li, Atharv Mittal et al.ICML 2026 · 3 citations
