Natural Adversarial Examples
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song
Abstract
Due to their complex nature, it is hard to characterize the ways in which machine learning models can misbehave or be exploited when deployed. Recent work on adversarial examples, i.e. inputs with minor perturbations that result in substantially different model predictions, is helpful in evaluating the robustness of these models by exposing the adversarial scenarios where they fail. However, these malicious perturbations are often unnatural, not semantically meaningful, and not applicable to complicated domains such as language. In this paper, we propose a framework to generate natural and legible adversarial examples that lie on the data manifold, by searching in semantic space of dense and continuous data representation, utilizing the recent advances in generative adversarial networks. We present generated adversaries to demonstrate the potential of the proposed approach for black-box classifiers for a wide range of applications such as image classification, textual entailment, and machine translation. We include experiments to show that the generated adversaries are natural, legible to humans, and useful in evaluating and analyzing black-box classifiers.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
616 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time cites Learning Transferable Visual Models From Natural Language Supervision
- A ConvNet for the 2020s cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
- Learning Transferable Visual Models From Natural Language Supervision cites The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
Cited by top-tier papers616
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- A ConvNet for the 2020sZhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer et al.CVPR 2022 · 6,782 citations
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution GeneralizationDan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath et al.ICCV 2021 · 2,294 citations
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timeMitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs et al.ICML 2022 · 1,464 citations
Related papers
- NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?Saadia Gabriel, Hamid Palangi, Yejin ChoiEMNLP Findings 2022
- Simulated Adversarial Testing of Face Recognition ModelsNataniel Ruiz, Adam Kortylewski, Weichao Qiu, Cihang Xie et al.CVPR 2022 · 8 citations
- Adversarial Text Generation by Search and LearningGuoyi Li, Bingkang Shi, Zongzhen Liu, Dehan Kong et al.EMNLP Findings 2023 · 14 citations
- Generating Natural Language Attacks in a Hard Label Black Box SettingRishabh Maheshwary, Saket Maheshwary, Vikram PudiAAAI 2021 · 128 citations
- URET: Universal Robustness Evaluation Toolkit (for Evasion)Kevin Eykholt, Taesung Lee, Douglas Lee Schales, Jiyong Jang et al.USENIX Security 2023
