Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
Minhao Cheng, Jinfeng Yi, Pin-Yu Chen, Huan Zhang, Cho-Jui Hsieh
Abstract
Crafting adversarial examples has become an important technique to evaluate the robustness of deep neural networks (DNNs). However, most existing works focus on attacking the image classification problem since its input space is continuous and output space is finite. In this paper, we study the much more challenging problem of crafting adversarial examples for sequence-to-sequence (seq2seq) models, whose inputs are discrete text strings and outputs have an almost infinite number of possibilities. To address the challenges caused by the discrete input space, we propose a projected gradient method combined with group lasso and gradient regularization. To handle the almost infinite output space, we design some novel loss functions to conduct non-overlapping attack and targeted keyword attack. We apply our algorithm to machine translation and text summarization tasks, and verify the effectiveness of the proposed algorithm: by changing less than 3 words, we can make seq2seq model to produce desired outputs with high success rates. We also use an external sentiment classifier to verify the property of preserving semantic meanings for our generated adversarial examples. On the other hand, we recognize that, compared with the well-evaluated CNN-based classifiers, seq2seq models are intrinsically more robust to adversarial attacks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
44 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers44
- TextBugger: Generating Adversarial Text Against Real-world ApplicationsJinfeng Li, Shouling Ji, Tianyu Du, Bo Li et al.NDSS 2019 · 876 citations
- Adversarial Sensor Attack on LiDAR-based Perception in Autonomous DrivingYulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou et al.CCS 2019 · 626 citations
- Robustness Verification for TransformersZhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang et al.ICLR 2020 · 131 citations
- Selective Question Answering under Domain ShiftAmita Kamath, Robin Jia, Percy LiangACL 2020 · 121 citations
- Adversarial Training with Fast Gradient Projection Method against Synonym Substitution Based Text AttacksXiaosen Wang, Yichen Yang, Yihe Deng, Kun HeAAAI 2021 · 98 citations
Related papers
- Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack FrameworkLifan Yuan, Yichi Zhang, Yangyi Chen, Wei WeiACL Findings 2023 · 27 citations
- TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven OptimizationBairu Hou, Jinghan Jia, Yihua Zhang, Guanhua Zhang et al.ICLR 2023 · 1 citation
- T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted AttackBoxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen et al.EMNLP 2020 · 55 citations
- Learning Optimization-based Adversarial Perturbations for Attacking Sequential Recognition ModelsXing Xu, Jiefu Chen, Jinhui Xiao, Zheng Wang et al.ACM MM 2020 · 18 citations
- Natural Language Induced Adversarial ImagesXiaopei Zhu, Peiyang Xu, Guanning Zeng, Yinpeng Dong et al.ACM MM 2024 · 1 citation
