EMNLP Findings2025
Dagger Behind Smile: Fool LLMs with a Happy Ending Story
Abstract
The wide adoption of Large Language Models (LLMs) has attracted significant attention from jailbreak attacks, where adversarial prompts crafted through optimization or manual design exploit LLMs to generate malicious contents. However, optimization-based attacks have limited efficiency and transferability, while existing manual designs are either easily detectable or demand intricate interactions with LLMs. In this paper, we first point out a novel perspective for jailbreak attacks: LLMs are more responsive to positive prompts. Based on this, we deploy Happy Ending Attack (HEA) to wrap up a malicious request in a scenario template involving a positive prompt formed mainly via a happy ending, it thus fools LLMs into jailbreaking either immediately or at a follow-up malicious request. This has made HEA both efficient and effective, as it requires only up to two turns to fully jailbreak LLMs. Extensive experiments show that our HEA can successfully jailbreak on state-of-the-art LLMs, including GPT-4o, Llama3-70b, Gemini-pro, and achieves 88.79% attack success rate on average. We also provide quantitative explanations for the success of HEA.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts cites Jailbroken: How Does LLM Safety Training Fail?
- Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Position cites Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs
- Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Position cites Jailbroken: How Does LLM Safety Training Fail?
- Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs cites Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
- Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues cites Jailbroken: How Does LLM Safety Training Fail?
Cited by top-tier papers2
Builds on10
- Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
- LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution ShiftsQibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie et al.ACL 2025 · 53 citations
- Interpreting Language Models with Contrastive ExplanationsKayo Yin, Graham NeubigEMNLP 2022 · 32 citations
- Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit CluesZhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang et al.ACL Findings 2024 · 23 citations
Related papers
- Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language ModelsZhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron et al.USENIX Security 2024 · 103 citations
- MAJIC: Markovian Adaptive Jailbreaking via Iterative Composition of Diverse Innovative StrategiesWeiwei Qi, Shuo Shao, Wei Gu, Tianhang Zheng et al.AAAI 2026
- Efficient LLM-Jailbreaking via Multimodal-LLM JailbreakHaoxuan Ji, Zheng Lin, Zhenxing Niu, Xinbo Gao et al.AAAI 2026 · 4 citations
- Stand on The Shoulders of Giants: Building JailExpert from Previous Attack ExperienceXi Wang, Songlei Jian, Shasha Li, Xiaopeng Li et al.EMNLP 2025 · 1 citation
- MASTERKEY: Automated Jailbreaking of Large Language Model ChatbotsGelei Deng, Yi Liu, Yuekang Li, Kailong Wang et al.NDSS 2024
