DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising
Zhenhao Li, Huichi Zhou, Marek Rei, Lucia Specia
Abstract
Pretrained language models have significantly advanced performance across various natural language processing tasks. However, adversarial attacks continue to pose a critical challenge to systems built using these models, as they can be exploited with carefully crafted adversarial texts. Inspired by the ability of diffusion models to predict and reduce noise in computer vision, we propose a novel and flexible adversarial defense method for language classification tasks, DiffuseDef 1 , which incorporates a diffusion layer as a denoiser between the encoder and the classifier. The diffusion layer is trained on top of the existing classifier, ensuring seamless integration with any model in a plug-and-play manner. During inference, the adversarial hidden state is first combined with sampled noise, then denoised iteratively and finally ensembled to produce a robust text representation. By integrating adversarial training, denoising, and ensembling techniques, we show that DiffuseDef improves over existing adversarial defense methods and achieves stateof-the-art performance against common blackbox and white-box adversarial attacks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Diffusion Models for Adversarial Purification cites Denoising Diffusion Probabilistic Models
- TextBugger: Generating Adversarial Text Against Real-world Applications cites Towards Evaluating the Robustness of Neural Networks
- Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment cites TextBugger: Generating Adversarial Text Against Real-world Applications
Builds on21
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Towards Evaluating the Robustness of Neural NetworksNicholas Carlini, David A. WagnerS&P 2017 · 9,786 citations
- Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
- TextBugger: Generating Adversarial Text Against Real-world ApplicationsJinfeng Li, Shouling Ji, Tianyu Du, Bo Li et al.NDSS 2019 · 876 citations
- Diffusion Models for Adversarial PurificationWeili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao et al.ICML 2022 · 663 citations
Related papers
- Text Adversarial Purification as Defense against Adversarial AttacksLinyang Li, Demin Song, Xipeng QiuACL 2023 · 11 citations
- Defending against Adversarial Audio via Diffusion ModelShutong Wu, Jiongxiao Wang, Wei Ping, Weili Nie et al.ICLR 2023 · 6 citations
- Textual Manifold-based Defense Against Natural Language Adversarial ExamplesDang Minh Nguyen, Anh Tuan LuuEMNLP 2022 · 13 citations
- Latent Diffusion for Language GenerationJustin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman et al.NeurIPS 2023 · 177 citations
- Searching for an Effective Defender: Benchmarking Defense against Adversarial Word SubstitutionZongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li et al.EMNLP 2021 · 46 citations
