Lune

ACL Findings2021

BERT-Defense: A Probabilistic Model Based on BERT to Combat Cognitively Inspired Orthographic Adversarial Attacks

Yannik Keller, Jan Mackensen, Steffen Eger

Abstract

Adversarial attacks expose important blind spots of deep learning systems. While wordand sentence-level attack scenarios mostly deal with finding semantic paraphrases of the input that fool NLP models, character-level attacks typically insert typos into the input stream. It is commonly thought that these are easier to defend via spelling correction modules. In this work, we show that both a standard spellchecker and the approach of Pruthi et al. (2019), which trains to defend against insertions, deletions and swaps, perform poorly on the character-level benchmark recently proposed in Eger and Benz (2020) which includes more challenging attacks such as visual and phonetic perturbations and missing word segmentations. In contrast, we show that an untrained iterative approach which combines context-independent character-level information with context-dependent information from BERT's masked language modeling can perform on par with human crowd-workers from Amazon Mechanical Turk (AMT) supervised via 3-shot learning.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factors cites BERT-ATTACK: Adversarial Attack Against BERT Using BERT
  • The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples cites BERT-ATTACK: Adversarial Attack Against BERT Using BERT
  • The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples cites Word-level Textual Adversarial Attacking as Combinatorial Optimization
  • The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples cites Robust Encodings: A Framework for Combating Adversarial Typos
  • The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples cites Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
  • Language Modelling with Pixels cites Robust Open-Vocabulary Translation from Visual Text Representations
  • SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher cites Word-level Textual Adversarial Attacking as Combinatorial Optimization
  • BERT-ATTACK: Adversarial Attack Against BERT Using BERT cites Word-level Textual Adversarial Attacking as Combinatorial Optimization
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert PatcherThai Le, Noseong Park, Dongwon LeeACL 2022 · 27 citations
  2. Language Modelling with PixelsPhillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky et al.ICLR 2023 · 17 citations
  3. Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic FactorsMarvin Kaster, Wei Zhao, Steffen EgerEMNLP 2021 · 13 citations
  4. The Best Defense is Attack: Repairing Semantics in Textual Adversarial ExamplesHeng Yang, Ke LiEMNLP 2024 · 3 citations

Builds on6

  1. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
  2. BERT-ATTACK: Adversarial Attack Against BERT Using BERTLinyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue et al.EMNLP 2020 · 529 citations
  3. Word-level Textual Adversarial Attacking as Combinatorial OptimizationYuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu et al.ACL 2020 · 188 citations
  4. Robust Encodings: A Framework for Combating Adversarial TyposErik Jones, Robin Jia, Aditi Raghunathan, Percy LiangACL 2020 · 92 citations
  5. Robust Open-Vocabulary Translation from Visual Text RepresentationsElizabeth Salesky, David Etter, Matt PostEMNLP 2021 · 33 citations

Related papers

  1. RMLM: A Flexible Defense Framework for Proactively Mitigating Word-level Adversarial AttacksZhaoyang Wang, Zhiyue Liu, Xiaopeng Zheng, Qinliang Su et al.ACL 2023 · 16 citations
  2. Evaluating the Robustness of Neural Language Models to Input PerturbationsMilad Moradi, Matthias SamwaldEMNLP 2021 · 64 citations
  3. Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text ModelsWenqiang Wang, Chongyang Du, Tao Wang, Kaihao Zhang et al.NeurIPS 2023 · 11 citations
  4. Text Adversarial Purification as Defense against Adversarial AttacksLinyang Li, Demin Song, Xipeng QiuACL 2023 · 11 citations
  5. Revisiting Character-level Adversarial Attacks for Language ModelsElías Abad-Rocamora, Yongtao Wu, Fanghui Liu, Grigorios Chrysos et al.ICML 2024 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines