ACL Findings2023
Adversarial Textual Robustness on Visual Dialog
Lu Yu, Verena Rieser
Abstract
Adversarial robustness evaluates the worst-case performance scenario of a machine learning model to ensure its safety and reliability. For example, cases where the user input contains a minimal change, e.g. a synonym, which causes the previously correct model to return a wrong answer. Using this scenario, this study is the first to investigate the robustness of visually grounded dialog models towards textual attacks. We first aim to understand how multimodal input components contribute to model robustness. Our results show that models which encode dialog history are more robust by providing redundant information. This is in contrast to prior work which finds that dialog history is negligible for model performance on this task. We also evaluate how to generate adversarial test examples which successfully fool the model but remain undetected by the user/software designer. Our analysis shows that the textual, as well as the visual context are important to generate plausible attacks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models cites Word-level Textual Adversarial Attacking as Combinatorial Optimization
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models cites Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
- BERT-ATTACK: Adversarial Attack Against BERT Using BERT cites Word-level Textual Adversarial Attacking as Combinatorial Optimization
Builds on14
- Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
- BERT-ATTACK: Adversarial Attack Against BERT Using BERTLinyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue et al.EMNLP 2020 · 529 citations
- On the Robustness of Vision Transformers to Adversarial ExamplesKaleel Mahmood, Rigel Mahmood, Marten van DijkICCV 2021 · 261 citations
- Word-level Textual Adversarial Attacking as Combinatorial OptimizationYuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu et al.ACL 2020 · 188 citations
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA ModelsLinjie Li, Jie Lei, Zhe Gan, Jingjing LiuICCV 2021 · 99 citations
Related papers
- Are aligned neural networks adversarially aligned?Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski et al.NeurIPS 2023 · 412 citations
- On the Robustness of Large Multimodal Models Against Image Adversarial AttacksXuanming Cui, Alejandro Aparcedo, Young Kyun Jang, Ser-Nam LimCVPR 2024
- Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial AttacksPeng Xie, Yequan Bie, Jianda Mao, Yangqiu Song et al.CVPR 2025
- Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language ModelsYubo Wang, Chaohu Liu, Yanqiu Qu, Haoyu Cao et al.ACM MM 2024 · 10 citations
- On the Robustness of Verbal Confidence of LLMs in Adversarial AttacksStephen Obadinma, Xiaodan ZhuNeurIPS 2025 · 3 citations
