Lune

ACL2025Top-tier venue

Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints

Junxiao Yang, Zhexin Zhang, Shiyao Cui, Hongning Wang, Minlie Huang

Abstract

Jailbreaking attacks can effectively induce unsafe behaviors in Large Language Models (LLMs); however, the transferability of these attacks across different models remains limited. This study aims to understand and enhance the transferability of gradient-based jailbreaking methods, which are among the standard approaches for attacking white-box models. Through a detailed analysis of the optimization process, we introduce a novel conceptual framework to elucidate transferability and identify superfluous constraints-specifically, the response pattern constraint and the token tail constraint-as significant barriers to improved transferability. Removing these unnecessary constraints substantially enhances the transferability and controllability of gradient-based attacks. Evaluated on Llama-3-8B-Instruct as the source model, our method increases the overall Transfer Attack Success Rate (T-ASR) across a set of target models with varying safety levels from 18.4% to 50.3%, while also improving the stability and controllability of jailbreak behaviors on both source and target models. Our code is available at https: //github.com/thu-coai/TransferAttack .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' Toxicity cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
  • Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites Toxicity in chatgpt: Analyzing persona-assigned language models
  • Tree of Attacks: Jailbreaking Black-Box LLMs Automatically cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
  • One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
  • Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization Invariance cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • AdvPrefix: An Objective for Nuanced LLM Jailbreaks cites Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
  • AdvPrefix: An Objective for Nuanced LLM Jailbreaks cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • AdvPrefix: An Objective for Nuanced LLM Jailbreaks cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • AdvPrefix: An Objective for Nuanced LLM Jailbreaks cites Safe RLHF: Safe Reinforcement Learning from Human Feedback
  • AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • Safe RLHF: Safe Reinforcement Learning from Human Feedback cites Toxicity in chatgpt: Analyzing persona-assigned language models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. AdvPrefix: An Objective for Nuanced LLM JailbreaksSicheng Zhu, Brandon Amos, Yuandong Tian, Chuan Guo et al.NeurIPS 2025 · 25 citations
  2. FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEctionRunqi Lin, Alasdair Paren, Suqin Yuan, Muyang Li et al.CVPR 2026 · 13 citations
  3. When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' ToxicityShiyao Cui, Xijia Feng, Yingkang Wang, Junxiao Yang et al.AAAI 2026
  4. One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMsYixin Tan, Yu Zhe, Rui Wen, Jun SakumaCCS 2026
  5. Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization InvarianceAo Wang, Xinghao Yang, Yongshun Gong, Wei Liu et al.ACL 2026

Builds on11

  1. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust RefusalMantas Mazeika, Long Phan, Xuwang Yin, Andy Zou et al.ICML 2024 · 1,031 citations
  2. Tree of Attacks: Jailbreaking Black-Box LLMs AutomaticallyAnay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson et al.NeurIPS 2024 · 835 citations
  3. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsXiaogeng Liu, Nan Xu, Muhao Chen, Chaowei XiaoICLR 2024 · 722 citations
  4. Safe RLHF: Safe Reinforcement Learning from Human FeedbackJosef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji et al.ICLR 2024 · 656 citations
  5. Toxicity in chatgpt: Analyzing persona-assigned language modelsAmeet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan et al.EMNLP Findings 2023 · 178 citations

Related papers

  1. Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness InstructionYuting Huang, Chengyuan Liu, Yifeng Feng, Yiquan Wu et al.ACL Findings 2025 · 1 citation
  2. Attention Eclipse: Manipulating Attention to Bypass LLM Safety-AlignmentPedram Zaree, Md Abdullah Al Mamun, Quazi Mishkatul Alam, Yue Dong et al.EMNLP 2025
  3. Improved Generation of Adversarial Examples Against Safety-aligned LLMsQizhang Li, Yiwen Guo, Wangmeng Zuo, Hao ChenNeurIPS 2024 · 23 citations
  4. Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy SpaceYao Huang, Yitong Sun, Shouwei Ruan, Yichi Zhang et al.ACL Findings 2025 · 8 citations
  5. TAO-Attack: Toward Advanced Optimization-Based Jailbreak Attacks for Large Language ModelsZhi Xu, Jiaqi Li, Xiaotong Zhang, Hong Yu et al.ICLR 2026 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines