EMNLP Findings2025
Forget the Unneeded: Backdooring Large Language Models via Contrastive-enhanced Machine Unlearning
Shiji Yang, Shu Zhao, Congyao Mei, Zhen Yang, Jie Chen, Fulan Qian, Zhen Duan, Yan-ping Zhang
Abstract
Prompt tuning for Large Language Models (LLMs) is vulnerable to backdoor attacks. Existing methods find backdoor attacks to be a significant threat in data-rich scenarios. However, in data-limited scenarios, these methods have difficulty capturing precise backdoor patterns, leading to weakened backdoor attack capabilities and significant side effects for the LLMs, which limits their practical relevance. To explore this problem, we propose a backdoor attacks through contrastive-enhanced machine unlearning in data-limited scenarios, called BCU. Specifically, BCU introduces a multiobjective machine unlearning method to capture precise backdoor patterns by forgetting the association between non-trigger data and the backdoor patterns, reducing side effects. Moreover, we design a contrastive learning strategy to enhance the association between triggers and backdoor patterns, improving the capability of backdoor attacks. Experimental results on 6 NLP datasets and 4 LLMs show that BCU exhibits strong backdoor attack capabilities and slight side effects, whether the training data is rich or limited. Our findings highlight practical security risks of backdoor attacks against LLMs, necessitating further research for security purposes. Our code is available at https://github.com/AHU-YangSJ/BCU.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- BadPrompt: Backdoor Attacks on Continuous Prompts cites AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
- BadPrompt: Backdoor Attacks on Continuous Prompts cites Language Models are Few-Shot Learners
- BadPrompt: Backdoor Attacks on Continuous Prompts cites Weight Poisoning Attacks on Pretrained Models
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts cites Language Models are Few-Shot Learners
- BadEdit: Backdooring Large Language Models by Model Editing cites BadPrompt: Backdoor Attacks on Continuous Prompts
- BadEdit: Backdooring Large Language Models by Model Editing cites Language Models are Few-Shot Learners
- BadEdit: Backdooring Large Language Models by Model Editing cites Weight Poisoning Attacks on Pretrained Models
Builds on16
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated PromptsTaylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace et al.EMNLP 2020 · 1,162 citations
- Weight Poisoning Attacks on Pretrained ModelsKeita Kurita, Paul Michel, Graham NeubigACL 2020 · 312 citations
- BadEdit: Backdooring Large Language Models by Model EditingYanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang et al.ICLR 2024 · 116 citations
- BadPrompt: Backdoor Attacks on Continuous PromptsXiangrui Cai, Haidong Xu, Sihan Xu, Ying Zhang et al.NeurIPS 2022 · 103 citations
Related papers
- BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIPJiawang Bai, Kuofeng Gao, Shaobo Min, Shu-Tao Xia et al.CVPR 2024
- UOR: Universal Backdoor Attacks on Pre-trained Language ModelsWei Du, Peixuan Li, Haodong Zhao, Tianjie Ju et al.ACL Findings 2024 · 10 citations
- ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language ModelsXuxu Liu, Siyuan Liang, Mengya Han, Yong Luo et al.ACL 2025 · 13 citations
- Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language ModelsPeihai Jiang, Xixiang Lyu, Yige Li, Jing MaAAAI 2025 · 8 citations
- Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language ModelsShuai Zhao, Jinming Wen, Anh Tuan Luu, Junbo Zhao et al.EMNLP 2023 · 39 citations
