Lune

CVPR Findings2026

Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality

Yanming Xiu, Zhengyuan Jiang, Neil Zhenqiang Gong, Maria Gorlatova

Abstract

Augmented reality (AR) has rapidly expanded over the past decade. As AR becomes increasingly integrated into daily life, its security and reliability emerge as critical challenges. Among various threats, contradictory virtual content attacks, where malicious or inconsistent virtual elements are introduced into the user's view, pose a unique risk by misleading users, creating semantic confusion, or delivering harmful information. In this work, we systematically model such attacks and present ContrAR, a novel benchmark for evaluating the robustness of vision-language models (VLMs) against virtual content manipulation and contradiction in AR. ContrAR contains 312 real-world AR videos validated by 10 human participants. We further benchmark 11 VLMs, including both commercial and opensource models. Experimental results reveal that while current VLMs exhibit reasonable understanding of contradictory virtual content, room still remains for improvement in detecting and reasoning about adversarial content manipulations in AR environments. Moreover, balancing detection accuracy and latency remains challenging.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on7

  1. The Dark Side of Perceptual Manipulations in Virtual RealityWen-Jie Tseng, Elise Bonnail, Mark McGill, Mohamed Khamis et al.CHI 2022 · 118 citations
  2. SEAR: Scaling Experiences in Multi-user Augmented RealityWenxiao Zhang, Bo Han, Pan HuiIEEE VR 2022 · 42 citations
  3. That Doesn't Go There: Attacks on Shared State in Multi-User Augmented Reality ApplicationsCarter Slocum, Yicheng Zhang, Erfan Shayegani, Pedram Zaree et al.USENIX Security 2024 · 21 citations
  4. ViDDAR: Vision Language Model-Based Task-Detrimental Content Detection for Augmented RealityYanming Xiu, Tim Scargill, Maria GorlatovaIEEE VR 2025 · 14 citations
  5. Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning ModelsQianqi Yan, Yue Fan, Hongquan Li, Shan Jiang et al.ACL Findings 2025 · 5 citations

Related papers

  1. MVTamperBench: Evaluating Robustness of Vision-Language ModelsAmit Agarwal, Srikant Panda, Angeline Charles, Hitesh Laxmichand Patel et al.ACL Findings 2025 · 6 citations
  2. MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMsHuiyi Chen, Jiawei Peng, Dehai Min, Changchang Sun et al.ICML 2026 · 18 citations
  3. Cognitive Attack Detection in Augmented Reality (CADAR): A Neuro-Symbolic Approach with Particle Filtering on Perception GraphsRongqian chen, Allison Andreyev, Yanming Xiu, Joshua Chilukuri, Shunav Sen, Mahdi Imani, Bin Li, Maria Gorlatova, Gang Tan, Tian LanCVPR Findings 2026
  4. Towards Visualization-of-Thought Jailbreak Attack against Large Visual Language ModelsHongqiong Zhong, Qingyang Teng, Baolin Zheng, Guanlin Chen et al.NeurIPS 2025 · 2 citations
  5. ProCap: Projection-Aware Captioning for Spatial Augmented RealityZimo Cao, Yuchen Deng, Haibin Ling, Bingyao HuangIEEE VR 2026
Dusk over the sea between two cliffs drawn in fine vertical lines