Lune

ACL2021Top-tier venue

A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger's Adversarial Attacks

Thai Le, Noseong Park, Dongwon Lee

Abstract

The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial textual attack method. Utilizing a learning-based mechanism, UniTrigger generates a fixed phrase that, when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. To defend against this attack that can cause significant harm, in this paper, we borrow the "honeypot" concept from the cybersecurity community and propose DARCY, a honeypot-based defense framework against UniTrigger. DARCY greedily searches and injects multiple trapdoors into an NN model to "bait and catch" potential attacks. Through comprehensive experiments across four public datasets, we show that DARCY detects UniTrigger's adversarial attacks with up to 99% TPR and less than 2% FPR in most cases, while maintaining the prediction accuracy (in F1) for clean inputs within a 1% margin. We also demonstrate that DARCY with multiple trapdoors is also robust to a diverse set of attack scenarios with attackers' varying levels of knowledge and skills. We release the source code of DARCY at: https://github.com/lethaiq/ ACL2021-DARCY-HoneypotDefenseNLP .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TextBugger: Generating Adversarial Text Against Real-world Applications cites Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
  • The Trickle-down Impact of Reward Inconsistency on RLHF cites TextShield: Beyond Successfully Detecting Adversarial Sentences in text classification
  • TextShield: Beyond Successfully Detecting Adversarial Sentences in text classification cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
  • TextShield: Beyond Successfully Detecting Adversarial Sentences in text classification cites SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher
  • On Robust Prefix-Tuning for Text Classification cites TextBugger: Generating Adversarial Text Against Real-world Applications
  • ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert PatcherThai Le, Noseong Park, Dongwon LeeACL 2022 · 27 citations
  2. On Robust Prefix-Tuning for Text ClassificationZonghan Yang, Yang LiuICLR 2022 · 23 citations
  3. The Trickle-down Impact of Reward Inconsistency on RLHFLingfeng Shen, Sihao Chen, Linfeng Song, Lifeng Jin et al.ICLR 2024 · 10 citations
  4. TextShield: Beyond Successfully Detecting Adversarial Sentences in text classificationLingfeng Shen, Ze Zhang, Haiyun Jiang, Ying ChenICLR 2023

Builds on4

  1. Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural NetworksBolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li et al.S&P 2019 · 1,801 citations
  2. TextBugger: Generating Adversarial Text Against Real-world ApplicationsJinfeng Li, Shouling Ji, Tianyu Du, Bo Li et al.NDSS 2019 · 876 citations
  3. ABS: Scanning Neural Networks for Back-doors by Artificial Brain StimulationYingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma et al.CCS 2019 · 531 citations
  4. Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial ExamplesMinhao Cheng, Jinfeng Yi, Pin-Yu Chen, Huan Zhang et al.AAAI 2020 · 268 citations

Related papers

  1. T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text ClassificationAhmadreza Azizi, Ibrahim Asadullah Tahmid, Asim Waheed, Neal Mangaokar et al.USENIX Security 2021 · 98 citations
  2. Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic TriggerFanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang et al.ACL 2021
  3. Gotta Catch'Em All: Using Honeypots to Catch Adversarial Attacks on Neural NetworksShawn Shan, Emily Wenger, Bolun Wang, Bo Li et al.CCS 2020 · 67 citations
  4. UNICORN: A Unified Backdoor Trigger Inversion FrameworkZhenting Wang, Kai Mei, Juan Zhai, Shiqing MaICLR 2023 · 7 citations
  5. CL-Attack: Textual Backdoor Attacks via Cross-Lingual TriggersJingyi Zheng, Tianyi Hu, Tianshuo Cong, Xinlei HeAAAI 2025 · 13 citations
Dusk over the sea between two cliffs drawn in fine vertical lines