Adaptive Instruction Composition for Automated LLM Red-Teaming
Jesse Zymet, Andy Luo, Swapnil Shinde, Sahil Wadhwa, Emily Chen
Abstract
Many approaches to LLM red-teaming leverage an attacker LLM to discover jailbreaks against a target. Several of them task the attacker with identifying effective strategies through trial and error, resulting in a semantically limited range of successes. Another approach discovers diverse attacks by combining crowdsourced harmful queries and tactics into instructions for the attacker, but does so at random, limiting effectiveness. This article introduces a novel framework, ADAPTIVE INSTRUCTION COM-POSITION, that combines crowdsourced texts according to an adaptive mechanism trained to jointly optimize effectiveness with diversity. We use reinforcement learning to balance exploration with exploitation in a combinatorial space of instructions to guide the attacker toward diverse generations tailored to target vulnerabilities. We demonstrate that our approach substantially outperforms random combination on a set of effectiveness and diversity metrics, even under model transfer. Further, we show that it surpasses a host of recent adaptive approaches on Harmbench. We employ a lightweight neural contextual bandit that adapts to contrastive embedding inputs, and provide ablations suggesting that the contrastive pretraining enables the network to rapidly generalize and scale to the massive space as it learns. Warning: this article discusses malicious content and methods for generating it using LLMs.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites Red Teaming Language Models with Language Models
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Builds on11
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust RefusalMantas Mazeika, Long Phan, Xuwang Yin, Andy Zou et al.ICML 2024 · 1,031 citations
- Tree of Attacks: Jailbreaking Black-Box LLMs AutomaticallyAnay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson et al.NeurIPS 2024 · 835 citations
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsXiaogeng Liu, Nan Xu, Muhao Chen, Chaowei XiaoICLR 2024 · 722 citations
- Neural Contextual Bandits with UCB-based ExplorationDongruo Zhou, Lihong Li, Quanquan GuICML 2020 · 329 citations
- Red Teaming Language Models with Language ModelsEthan Perez, Saffron Huang, H. Francis Song, Trevor Cai et al.EMNLP 2022 · 239 citations
Related papers
- Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA ExpertsChristos Ziakas, Nicholas Loo, Nishita Jain, Alessandra RussoACL 2026 · 2 citations
- AdvPrompter: Fast Adaptive Adversarial Prompting for LLMsAnselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos et al.ICML 2025
- WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language ModelsLiwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger et al.NeurIPS 2024 · 247 citations
- Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained OptimizationKai Hu, Weichen Yu, Yining Li, Tianjun Yao et al.NeurIPS 2024 · 31 citations
- Active Attacks: Red-teaming LLMs via Adaptive EnvironmentsTaeyoung Yun, Pierre-Luc St-Charles, Jinkyoo Park, Yoshua Bengio et al.ICML 2026 · 5 citations
