Lune

EMNLP Findings2025

Overcoming Black-box Attack Inefficiency with Hybrid and Dynamic Select Algorithms

Abhinay Shankar Belde, Rohit Ramkumar, Jonathan Rusert

Abstract

Adversarial text attack research plays a crucial role in evaluating the robustness of NLP models. However, the increasing complexity of transformer-based architectures has dramatically raised the computational cost of attack testing, especially for researchers with limited resources (e.g., GPUs). Existing popular black-box attack methods often require a large number of queries, which can make them inefficient and impractical for researchers. To address these challenges, we propose two new attack selection strategies called Hybrid and Dynamic Select, which better combine the strengths of previous selection algorithms. Hybrid Select merges generalized BinarySelect techniques with GreedySelect by introducing a size threshold to decide which selection algorithm to use. Dynamic Select provides an alternative approach of combining the generalized Binary and GreedySelect by learning which lengths of texts each selection method should be applied to. This greatly reduces the number of queries needed while maintaining attack effectiveness (a limitation of BinarySelect). Across 4 datasets and 6 target models, our best method(sentence-level Hybrid Select) is able to reduce the number of required queries per attack up 25.82% on average against both encoder models and LLMs, without losing the effectiveness of the attack.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • BERT-ATTACK: Adversarial Attack Against BERT Using BERT cites TextBugger: Generating Adversarial Text Against Real-world Applications
  • Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment cites TextBugger: Generating Adversarial Text Against Real-world Applications
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
  2. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and EntailmentDi Jin, Zhijing Jin, Joey Tianyi Zhou, Peter SzolovitsAAAI 2020 · 1,333 citations
  3. TextBugger: Generating Adversarial Text Against Real-world ApplicationsJinfeng Li, Shouling Ji, Tianyu Du, Bo Li et al.NDSS 2019 · 876 citations
  4. BERT-ATTACK: Adversarial Attack Against BERT Using BERTLinyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue et al.EMNLP 2020 · 529 citations
  5. Making Monolingual Sentence Embeddings Multilingual using Knowledge DistillationNils Reimers, Iryna GurevychEMNLP 2020 · 54 citations

Related papers

  1. Adversarial Text Generation by Search and LearningGuoyi Li, Bingkang Shi, Zongzhen Liu, Dehan Kong et al.EMNLP Findings 2023 · 14 citations
  2. Improving the Robustness of Transformer-based Large Language Models with Dynamic AttentionLujia Shen, Yuwen Pu, Shouling Ji, Changjiang Li et al.NDSS 2024
  3. A Strong Baseline for Query Efficient Attacks in a Black Box SettingRishabh Maheshwary, Saket Maheshwary, Vikram PudiEMNLP 2021 · 38 citations
  4. Multi-granularity Textual Adversarial Attack with Behavior CloningYangyi Chen, Jin Su, Wei WeiEMNLP 2021 · 29 citations
  5. How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacksSalijona Dyrmishi, Salah Ghamizi, Maxime CordyACL 2023 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines