Meerkat-VL: Implicit Risk Safety Alignment in Multimodal LLMs via Perceptual Reasoning and Self-Verification
Peicheng Zhou, Chuanbin Liu, Shancheng Fang, Bowei Pu, Yiwei Sun, Zhangchi Hu, Hongtao Xie
Abstract
Multimodal LLMs (MLLMs) are increasingly deployed across diverse applications, but they pose significant safety concerns due to cross-modal interactions. To improve model safety awareness, existing methods rely on explicit-risk preference datasets and reinforcement learning guided by safety rewards. While effective in improving models' safety awareness, these methods still face data scarcity and reward hacking in implicit-risk scenarios, leading to insufficient risk perception and harmful responses. To address these challenges, we propose Meerkat-VL, a framework that enables models to perceive and verify implicit risks while generating safe responses. First, we introduce Meerkat-Safe, the first training dataset with detailed labels for implicit risks. Second, we develop Normative Perceptual Self-Verification, which enables models to verify both perceptual reasoning and responses. This process provides denser and more reliable rewards for perception accuracy and answer safety, thereby mitigating reward hacking. Finally, we propose Dual-Objective Perceptual Consistency Alignment, encouraging models to generate safe responses by penalizing answers that follow safe templates without accurate risk perception. Extensive experiments show that Meerkat-VL consistently outperforms baselines on multimodal safety benchmarks, improving safety and helpfulness by 16% and 13%, and achieving a 32% safety gain on implicit-risk tasks. Our codes are available at https://github.com/Tunanzzz/Meerkat-VL.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext bd13a2c4-ca1e-4ebe-8ee7-af95fa72e846Builds on21
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong et al.NeurIPS 2023 · 4,013 citations
- Self-Instruct: Aligning Language Models with Self-Generated InstructionsYizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu et al.ACL 2023 · 540 citations
- MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language BenchmarkDongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang et al.ICML 2024 · 345 citations
Related papers
- Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMsMing Wen, Kun Yang, Xin Chen, Jingyu Zhang et al.ICLR 2026 · 4 citations
- Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human FeedbackJiaming Ji, Xinyu Chen, Rui Pan, Han Zhu et al.NeurIPS 2025 · 28 citations
- Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related ImagesQishun Yang, Shu Yang, Lijie Hu, Di WangACL 2026 · 1 citation
- SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language ModelsYongting Zhang, Lu Chen, Guodong Zheng, Yifeng Gao et al.CVPR 2025
- Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMsWenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang et al.ACL 2025
