ACL Findings2026
Incomplete Prompt Jailbreaks in Large Language Models
Abstract
Large language models (LLMs) are increasingly released as open-weight models with safeguards against harmful requests. Nevertheless, sentence completion remains vulnerable to incomplete harmful prompts. In this work, we formalize this phenomenon as incomplete prompt jailbreaks (IPJ) and provide a systematic empirical characterization of when and how incomplete prompts elicit harmful continuations. We analyze diverse attractor types associated with incomplete sentence continuation and show that LLMs systematically delay refusal until sentence termination. We further demonstrate that training models to refuse incomplete harmful prompts via parameter tuning is insufficient, failing to generalize across both content domains and attractor types. To enable fine-grained control, we identify two functional neurons: termination and continuation neurons. By clarifying their roles in sentence completion, we highlight the potential of neuron-level interventions for more precise and robust IPJ defenses.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! cites Training language models to follow instructions with human feedback
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
- Jailbroken: How Does LLM Safety Training Fail? cites Training language models to follow instructions with human feedback
Builds on21
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
Related papers
- Refusal in Language Models Is Mediated by a Single DirectionAndy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka et al.NeurIPS 2024 · 1,166 citations
- Defending LLMs against Jailbreaking Attacks via BacktranslationYihan Wang, Zhouxing Shi, Andrew Bai, Cho-Jui HsiehACL Findings 2024 · 28 citations
- Interpreting the Repeated Token Phenomenon in Large Language ModelsItay Yona, Ilia Shumailov, Jamie Hayes, Yossi GandelsmanICML 2025
- Neuronal Insights into LLM Attacks: Targeted Neuron Tuning for Precise and Robust Vulnerability PatchingDan Shi, Renren Jin, Zhuowen Han, Yuqi Ren et al.ACL Findings 2026
- TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process RewardsXiqiao Xiong, Ouxiang Li, Zhuo Liu, Moxin Li et al.ACL 2026 · 7 citations
