ACL Findings2024
Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues
Zhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang, Qing Wang, Yang Liu
Abstract
With the development of LLMs, the security threats of LLMs are getting more and more attention. Numerous jailbreak attacks have been proposed to assess the security defense of LLMs. Current jailbreak attacks primarily utilize scenario camouflage techniques. However, their explicit mention of malicious intent will be easily recognized and defended by LLMs. In this paper, we propose an indirect jailbreak attack approach, Puzzler, which can bypass the LLM's defensive strategies and obtain malicious responses by implicitly providing LLMs with some clues about the original malicious query. In addition, inspired by the wisdom of "When unable to attack, defend" from Sun Tzu's Art of War, we adopt a defensive stance to gather clues about the original malicious query through LLMs. The experimental results indicate that the Query Success Rate of the Puzzler is 14.0%-82.7% higher than baselines on the most prominent LLMs. Furthermore, when tested against the state-of-the-art jailbreak detection approaches, Puzzler proves to be more effective at evading detection compared to baselines.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
27 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs cites NeuroStrike: Neuron-Level Attacks on Aligned LLMs
- GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs cites Language Models are Few-Shot Learners
- GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs cites Jailbroken: How Does LLM Safety Training Fail?
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- Sok: Evaluating Jailbreak Guardrails for Large Language Models cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
- Sok: Evaluating Jailbreak Guardrails for Large Language Models cites When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search
- Sok: Evaluating Jailbreak Guardrails for Large Language Models cites Jailbroken: How Does LLM Safety Training Fail?
- Sok: Evaluating Jailbreak Guardrails for Large Language Models cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
- When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- NeuroStrike: Neuron-Level Attacks on Aligned LLMs cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
- NeuroStrike: Neuron-Level Attacks on Aligned LLMs cites Jailbroken: How Does LLM Safety Training Fail?
- NeuroStrike: Neuron-Level Attacks on Aligned LLMs cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- Jailbroken: How Does LLM Safety Training Fail? cites Language Models are Few-Shot Learners
Cited by top-tier papers27
- PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration TestingGelei Deng, Yi Liu, Víctor Mayoral Vilches, Peng Liu et al.USENIX Security 2024 · 186 citations
- When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided SearchXuan Chen, Yuzhou Nie, Wenbo Guo, Xiangyu ZhangNeurIPS 2024 · 68 citations
- Sok: Evaluating Jailbreak Guardrails for Large Language ModelsXunguang Wang, Zhenlan Ji, Wenxuan Wang, Zongjie Li et al.S&P 2026 · 27 citations
- NeuroStrike: Neuron-Level Attacks on Aligned LLMsLichao Wu, Sasha Behrouzi, Mohamadreza Rostami, Maximilian Thang et al.NDSS 2026 · 21 citations
- GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMsLichao Wu, Sasha Behrouzi, Mohamadreza Rostami, Stjepan Picek et al.USENIX Security 2026 · 14 citations
Builds on6
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations
- LIMA: Less Is More for AlignmentChunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer et al.NeurIPS 2023 · 1,486 citations
- Tree of Attacks: Jailbreaking Black-Box LLMs AutomaticallyAnay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson et al.NeurIPS 2024 · 835 citations
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsXiaogeng Liu, Nan Xu, Muhao Chen, Chaowei XiaoICLR 2024 · 722 citations
Related papers
- Efficient LLM-Jailbreaking via Multimodal-LLM JailbreakHaoxuan Ji, Zheng Lin, Zhenxing Niu, Xinbo Gao et al.AAAI 2026 · 4 citations
- JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMsJunjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen et al.ACL 2025
- RobustKV: Defending Large Language Models against Jailbreak Attacks via KV EvictionTanqiu Jiang, Zian Wang, Jiacheng Liang, Changjiang Li et al.ICLR 2025
- Jailbreak LLMs through Internal Stance ManipulationShuangjie Fu, Du Su, Beining Huang, Fei Sun et al.EMNLP 2025
- BaitAttack: Alleviating Intention Shift in Jailbreak Attacks via Adaptive Bait CraftingRui Pu, Chaozhuo Li, Rui Ha, Litian Zhang et al.EMNLP 2024
