ACL Findings2026
VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization
Menglan Chen, Xianghe Pang, Jingjing Dong, Wenhao Wang, Yaxin Du, Siheng Chen
Abstract
Aligning Vision-Language Models (VLMs) with safety standards is essential to mitigate risks arising from their multimodal complexity, where integrating vision and language unveils subtle threats beyond the reach of conventional safeguards. Inspired by the insight that reasoning across modalities is key to preempting intricate vulnerabilities, we propose a novel direction for VLM safety: multimodal reasoning-driven prompt rewriting. To this end, we introduce VLMGuard-R1, a proactive framework that refines user inputs through a reasoning-guided rewriter, dynamically interpreting text-image interactions to deliver refined prompts that bolster safety across diverse VLM architectures without altering their core parameters. To achieve this, we devise a threestage reasoning pipeline to synthesize a dataset that trains the rewriter to infer subtle threats, enabling tailored, actionable responses over generic refusals. Extensive experiments across five benchmarks with six VLMs reveal that VLMGuard-R1 outperforms four baselines. In particular, VLMGuard-R1 achieves a remarkable 43.59% increase in average safety across five models on the SIUO benchmark.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Language Models are Few-Shot Learners
- GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models cites Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models
- GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models cites FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
- EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment cites Visual Instruction Tuning
- EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment cites Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models
- Visual Instruction Tuning cites Language Models are Few-Shot Learners
- Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models cites Visual Instruction Tuning
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts cites Visual Instruction Tuning
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts cites Language Models are Few-Shot Learners
Cited by top-tier papers2
Builds on12
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual PromptsYichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang et al.AAAI 2025 · 350 citations
- Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language ModelsYongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang et al.ICML 2024 · 140 citations
Related papers
- Iterative Prompt Refinement for Safer Text-to-Image GenerationJinwoo Jeon, JunHyeok Oh, Hayeong Lee, Byung-Jun LeeEMNLP 2025
- Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image GenerationHongji Yang, Yucheng Zhou, Wencheng Han, Jianbing ShenACL Findings 2025
- ELITE: Enhanced Language-Image Toxicity Evaluation for SafetyWonjun Lee, Doehyeon Lee, Eugene Choi, Sangyoon Yu et al.ICML 2025
- Automated Red Teaming for Text-to-Image Models Through Feedback-Guided Prompt Iteration with Vision-Language ModelsWei Xu, Kangjie Chen, Jiawei Qiu, Yuyang Zhang et al.ICCV 2025 · 3 citations
- From Evaluation to Defense: Advancing Safety in Video Large Language ModelsYiwei Sun, Peiqi Jiang, Chuanbin Liu, Luohao Lin et al.ICLR 2026 · 2 citations
