Lune

CVPR Findings2026

When Harmful Content Goes Invisible: Unveiling Perception Failure of LVLMs with CAMOUHARMTI

Yanhui Li, Qi Zhou, Zhihong Xu, Huizhong Guo, Wenhai Wang, Dongxia Wang

Abstract

Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial, such as online content moderation. However, real-world harmful content is often camouflaged, relying on nuanced text-image interplay, such as memes or images with embedded malicious text, to evade detection. This raises a key question: can LVLMs perceive such camouflaged harmful content as sensitively as humans do? In this paper, we introduce CAMHARMTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions. CAMHARMTI consists of over 4,500 samples across three types of image-text posts. Experiments on 100 human users and 12 mainstream LVLMs reveal a clear perceptual gap: humans easily recognize such content (e.g., over 95.75% accuracy), whereas current LVLMs often fail (e.g., ChatGPT-4o achieves only 2.10% accuracy). Moreover, fine-tuning experiments demonstrate that CAMHARMTI serves as an effective resource for improving model perception, increasing accuracy by 55.94% for Qwen2.5VL-7B. Attention analysis and layer-wise probing further reveal that fine-tuning enhances sensitivity primarily in the early layers of the vision encoder, promoting a more integrated scene understanding. These findings highlight the inherent perceptual limitations in LVLMs and offer insight into more humanaligned visual reasoning systems. Dataset available here.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  2. Adversarial Illusions in Multi-Modal EmbeddingsTingwei Zhang, Rishi D. Jha, Eugene Bagdasaryan, Vitaly ShmatikovUSENIX Security 2024 · 32 citations
  3. IllusionCAPTCHA: A CAPTCHA based on Visual IllusionZiqi Ding, Gelei Deng, Yi Liu, Junchen Ding et al.WWW 2025 · 15 citations
  4. Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path ForwardZhiyuan Fan, Yumeng Wang, Sandeep Polisetty, Yi R. FungACL Findings 2025 · 4 citations
  5. Improved Baselines with Visual Instruction TuningHaotian Liu, Chunyuan Li, Yuheng Li, Yong Jae LeeCVPR 2024

Related papers

  1. Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content ModerationZhiheng Li, Zongyang Ma, Yuntong Pan, Ziqi Zhang et al.ACL Findings 2026
  2. Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMsWenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang et al.ACL 2025
  3. MVP-Bench: Can Large Vision-Language Models Conduct Multi-level Visual Perception Like Humans?Guanzhen Li, Yuxi Xie, Min-Yen KanEMNLP Findings 2024 · 1 citation
  4. ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs: ShieldVLMShiyao Cui, Qinglin Zhang, Xuan Ouyang, Renmiao Chen et al.ACM MM 2025 · 3 citations
  5. Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language ModelsYi Ding, Lijun Li, Bing Cao, Jing ShaoICLR 2026 · 21 citations
Dusk over the sea between two cliffs drawn in fine vertical lines