Lune

EMNLP Findings2025

AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders

Yuqi Zhang, Yuchun Miao, Zuchao Li, Liang Ding

Abstract

We introduce AMIA, a lightweight, inferenceonly defense for Large Vision-Language Models (LVLMs) that (1) Automatically Masks a small set of text-irrelevant image patches to disrupt adversarial perturbations, and (2) conducts joint Intention Analysis to uncover and mitigate hidden harmful intents before response generation. Without any retraining, AMIA improves defense success rates across diverse LVLMs and jailbreak benchmarks from an average of 52.4% to 81.7%, preserves general utility with only a 2% average accuracy drop, and incurs only modest inference overhead. Ablation confirms both masking and intention analysis are essential for a robust safety-utility trade-off.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs cites High-Resolution Image Synthesis with Latent Diffusion Models
  • GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models cites FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
  • Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency cites FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. GuardAlign: Test-time Safety Alignment in Multimodal Large Language ModelsXingyu Zhu, Beier Zhu, Junfeng Fang, Shuo Wang et al.ICLR 2026 · 2 citations

Builds on8

  1. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  2. FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual PromptsYichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang et al.AAAI 2025 · 350 citations
  3. Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMsShengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma et al.CVPR 2024 · 111 citations
  4. InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward ModelingYuchun Miao, Sen Zhang, Liang Ding, Rong Bao et al.NeurIPS 2024 · 108 citations
  5. Jailbreaking Multimodal Large Language Models via Shuffle InconsistencyShiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen et al.ICCV 2025 · 8 citations

Related papers

  1. How Jailbreak Defenses Work and Ensemble? A Mechanistic InvestigationZhuohan Long, Siyuan Wang, Shujun Liu, Yuhang LaiEMNLP Findings 2025 · 1 citation
  2. MetaDefense: Defending Fine-tuning based Jailbreak Attack Before and During GenerationWeisen Jiang, Sinno Jialin PanNeurIPS 2025 · 10 citations
  3. One Head to Rule Them All: Amplifying LVLM Safety through a Single Critical Attention HeadJunhao Xia, Haotian Zhu, Shuchao Pang, Zhigang Lu et al.NeurIPS 2025 · 5 citations
  4. Attack as Defense: Safeguarding Large Vision-Language Models from Jailbreaking by Adversarial AttacksChongxin Li, Hanzhang Wang, Yuchun FangEMNLP Findings 2025 · 1 citation
  5. Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against JailbreaksHan Wang, Gang Wang, Huan ZhangCVPR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines