EMNLP Findings2025
MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?
Muntasir Wahed, Xiaona Zhou, Kiet A. Nguyen, Tianjiao Yu, Nirav Diwan, Gang Wang, Dilek Hakkani-Tür, Ismini Lourentzou
Abstract
Recent advancements in LLMs have significantly enhanced code generation.However, their robustness against multi-turn malicious coding jailbreaks remains underexplored.In this work, we introduce code decomposition attacks, where a malicious coding task is disguised as a sequence of benign subtasks across multiple conversational turns to bypass safety filters.To facilitate systematic evaluation, we curate MOCHA, a large-scale benchmark of 10K single-turn and multi-turn jailbreak coding prompts spanning 13 real-world threat categories.Experiments on open-and closedsource models reveal persistent vulnerabilities, particularly in multi-turn settings where rejection rates drop sharply.Fine-tuning on MOCHA improves robustness while preserving coding ability, and generalizes to external adversarial datasets, yielding up to 32.4% increase in rejection rates without additional supervision.* This paper was developed by a subset of the UIUC team that participated in the Amazon NOVA AI Challenge.For a comprehensive account of the full team effort, see the accompanying Amazon technical report (Liu et al., 2025a).45 50 55 60 65 70 75 80 85 Pass@1 (%)
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- When "Correct" Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents? cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation cites CodeT5+: Open Code Large Language Models for Code Understanding and Generation
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Cited by top-tier papers1
Builds on22
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code GenerationJiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming ZhangNeurIPS 2023 · 2,317 citations
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust RefusalMantas Mazeika, Long Phan, Xuwang Yin, Andy Zou et al.ICML 2024 · 1,031 citations
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsXiaogeng Liu, Nan Xu, Muhao Chen, Chaowei XiaoICLR 2024 · 722 citations
- CodeT5+: Open Code Large Language Models for Code Understanding and GenerationYue Wang, Hung Le, Akhilesh Gotmare, Nghi D. Q. Bui et al.EMNLP 2023 · 339 citations
Related papers
- LLMs Caught in the Crossfire: Malware Requests and Jailbreak ChallengesHaoyang Li, Huan Gao, Zhiyuan Zhao, Zhiyu Lin et al.ACL 2025
- Chain of Attack: Hide Your Intention through Multi-Turn InterrogationXikang Yang, Biyu Zhou, Xuehai Tang, Jizhong Han et al.ACL Findings 2025 · 3 citations
- MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM SafetyJialin Song, Xiaodong Liu, Weiwei Yang, Wuyang Chen et al.ICML 2026 · 5 citations
- Exploiting Task-Level Vulnerabilities: An Automatic Jailbreak Attack and Defense Benchmarking for LLMsLan Zhang, Xinben Gao, Liuyi Yao, Jinke Song et al.USENIX Security 2025
- Analogy-based Multi-Turn Jailbreak against Large Language ModelsMengjie Wu, Yihao Huang, Zhenjun Lin, Kangjie Chen et al.NeurIPS 2025 · 9 citations
