ACL Findings2025
MVTamperBench: Evaluating Robustness of Vision-Language Models
Amit Agarwal, Srikant Panda, Angeline Charles, Hitesh Laxmichand Patel, Bhargava Kumar, Priyaranjan Pattnayak, Taki Hasan Rafi, Tejaswini Kumar, Hansa Meghwani, Karan Gupta, Dong-Kyu Chae
Abstract
Multimodal Large Language Models (MLLMs), are recent advancement of Vision-Language Models (VLMs) that have driven major advances in video understanding. However, their vulnerability to adversarial tampering and manipulations remains underexplored. To address this gap, we introduce MVTamperBench, a benchmark that systematically evaluates MLLM robustness against five prevalent tampering techniques: rotation, masking, substitution, repetition, and dropping; based on real-world visual tampering scenarios such as surveillance interference, social media content edits, and misinformation injection. MVTamperBench comprises 3.4K original videos, expanded into over 17K tampered clips covering 19 distinct video manipulation tasks. This benchmark challenges models to detect manipulations in spatial and temporal coherence. We evaluate 45 recent MLLMs from 15+ model families. We reveal substantial variability in resilience across tampering types and show that larger parameter counts do not necessarily guarantee robustness. MVTamperBench sets a new benchmark for developing tamper-resilient MLLM in safety-critical applications, including detecting clickbait, preventing harmful content distribution, and enforcing policies on media platforms. We release all code, data, and benchmark to foster open research in trustworthy video understanding. labelling for key-value extraction from documents. US Patent 11,823,478.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on18
- LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic AlignmentBin Zhu, Bin Lin, Munan Ning, Yang Yan et al.ICLR 2024 · 403 citations
- Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language ModelsYushi Hu, Weijia Shi, Xingyu Fu, Dan Roth et al.NeurIPS 2024 · 373 citations
- Paxion: Patching Action Knowledge in Video-Language Foundation ModelsZhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu et al.NeurIPS 2023 · 54 citations
- Cross-Modal Transferable Adversarial Attacks from Images to VideosZhipeng Wei, Jingjing Chen, Zuxuan Wu, Yu-Gang JiangCVPR 2022 · 45 citations
- LongVILA: Scaling Long-Context Visual Language Models for Long VideosYukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu et al.ICLR 2025 · 1 citation
Related papers
- Drifting Away from Truth: GenAI-Driven News Diversity Challenges LVLM-Based Misinformation DetectionFanxiao Li, Jiaying Wu, Tingchao Fu, Yunyun Dong et al.AAAI 2026 · 4 citations
- MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMsHuiyi Chen, Jiawei Peng, Dehai Min, Changchang Sun et al.ICML 2026 · 18 citations
- From Pixels to Semantics: A Novel MLLM-Driven Approach for Explainable Tampered Text DetectionGuitao Xu, Ziqi Yi, Peirong Zhang, Jiahuan Cao et al.ACM MM 2025 · 2 citations
- Editing the Moving World: Model Editing for Video LLMsQian Zhang, Xinye Li, Xiaokai Wu, Junhao Xu et al.ACL 2026
- VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language ModelsRohit Saxena, Alessandro Suglia, Pasquale MinerviniICML 2026 · 7 citations
