Lune

ACL Findings2023

TextVerifier: Robustness Verification for Textual Classifiers with Certifiable Guarantees

Siqi Sun, Wenjie Ruan

Abstract

When textual classifiers are deployed in safetycritical workflows, they must withstand the onslaught of AI-enabled model confusion caused by adversarial examples with minor alterations.In this paper, the main objective is to provide a formal verification framework, called TextVerifier, with certifiable guarantees on deep neural networks in natural language processing against word-level alteration attacks.We aim to provide an approximation of the maximal safe radius by deriving provable bounds both mathematically and automatically, where a minimum word-level L 0 distance is quantified as a guarantee for the classification invariance of victim models.Here, we illustrate three strengths of our strategy: i) certifiable guarantee: effective verification with convergence to ensure approximation of maximal safe radius with tight bounds ultimately; ii) high-efficiency: it yields an efficient speed edge by a novel parallelization strategy that can process a set of candidate texts simultaneously on GPUs; and iii) reliable anytime estimation: the verification can return intermediate bounds, and robustness estimates that are gradually, but strictly, improved as the computation proceeds.Furthermore, experiments are conducted on text classification on four datasets over three victim models to demonstrate the validity of tightening bounds.Our tool TextVerifier is available at https: //github.com/TrustAI/TextVerifer.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • BERT-ATTACK: Adversarial Attack Against BERT Using BERT cites TextBugger: Generating Adversarial Text Against Real-world Applications
  • BERT-ATTACK: Adversarial Attack Against BERT Using BERT cites Word-level Textual Adversarial Attacking as Combinatorial Optimization
  • Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment cites TextBugger: Generating Adversarial Text Against Real-world Applications
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Certified Robustness Under Bounded Levenshtein DistanceElías Abad-Rocamora, Grigorios Chrysos, Volkan CevherICLR 2025

Builds on12

  1. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
  2. TextBugger: Generating Adversarial Text Against Real-world ApplicationsJinfeng Li, Shouling Ji, Tianyu Du, Bo Li et al.NDSS 2019 · 876 citations
  3. BERT-ATTACK: Adversarial Attack Against BERT Using BERTLinyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue et al.EMNLP 2020 · 529 citations
  4. MACER: Attack-free and Scalable Robust Training via Maximizing Certified RadiusRuntian Zhai, Chen Dan, Di He, Huan Zhang et al.ICLR 2020 · 195 citations
  5. Word-level Textual Adversarial Attacking as Combinatorial OptimizationYuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu et al.ACL 2020 · 188 citations

Related papers

  1. Assessing Robustness of Text Classification through Maximal Safe Radius ComputationEmanuele La Malfa, Min Wu, Luca Laurenti, Benjie Wang et al.EMNLP Findings 2020 · 10 citations
  2. Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial AttacksXinyu Zhang, Hanbin Hong, Yuan Hong, Peng Huang et al.S&P 2024 · 41 citations
  3. Scalable Quantitative Verification For Deep Neural NetworksTeodora Baluta, Zheng Leong Chua, Kuldeep S. Meel, Prateek SaxenaICSE 2021 · 39 citations
  4. Towards General Robustness Verification of MaxPool-Based Convolutional Neural Networks via Tightening Linear ApproximationYuan Xiao, Shiqing Ma, Juan Zhai, Chunrong Fang et al.CVPR 2024 · 1 citation
  5. Certified Robustness to Programmable Transformations in LSTMsYuhao Zhang, Aws Albarghouthi, Loris D'AntoniEMNLP 2021 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines