Lune

USENIX Security2025Top-tier venue

Refusal Is Not an Option: Unlearning Safety Alignment of Large Language Models

Minkyoo Song, Hanna Kim, Jaehan Kim, Seungwon Shin, Sooel Son

Abstract

Safety alignment has become an indispensable procedure to ensure the safety of large language models (LLMs), as they are reported to generate harmful, privacy-sensitive, and copyrighted content when prompted with adversarial instructions. Machine unlearning is a representative approach to establishing the safety of LLMs, enabling them to forget problematic training instances and thereby minimize their influence. However, no prior study has investigated the feasibility of adversarial unlearning-using seemingly legitimate unlearning requests to compromise the safety of a target LLM. In this paper, we introduce novel attack methods designed to break LLM safety alignment through unlearning. The key idea lies in crafting unlearning instances that cause the LLM to forget its mechanisms for rejecting harmful instructions. Specifically, we propose two attack methods. The first involves explicitly extracting rejection responses from the target LLM and feeding them back for unlearning. The second attack exploits LLM agents to obscure rejection responses by merging them with legitimate-looking unlearning requests, increasing their chances of bypassing internal filtering systems. Our evaluations show that these attacks significantly compromise the safety of two open-source LLMs: LLaMA and Phi. LLaMA's harmfulness scores increase by an average factor of 11 across four representative unlearning methods, while Phi exhibits a 61.8× surge in the rate of unsafe responses. Furthermore, we demonstrate that our unlearning attack is also effective against OpenAI's fine-tuning service, increasing GPT-4o's harmfulness score by 2.21×. Our work identifies a critical vulnerability in unlearning and represents an important first step toward developing safe and responsible unlearning practices while honoring users' unlearning requests. Our code is

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
  • The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning cites Measuring Massive Multitask Language Understanding
  • The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on24

  1. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  2. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  4. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
  5. The WMDP Benchmark: Measuring and Reducing Malicious Use with UnlearningNathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue et al.ICML 2024 · 390 citations

Related papers

  1. Invisible Safety Threat: Malicious Finetuning for LLM via SteganographyGuangnian Wan, Xinyin Ma, Gongfan Fang, Xinchao WangICLR 2026 · 4 citations
  2. Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine UnlearningYiwei Chen, Yuguang Yao, Yihua Zhang, Bingquan Shen et al.ICLR 2026
  3. Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!Zhanhui Zhou, Jie Liu, Zhichen Dong, Jiaheng Liu et al.ACL 2024
  4. On the Vulnerability of Safety Alignment in Open-Access LLMsJingwei Yi, Rui Ye, Qisi Chen, Bin Zhu et al.ACL Findings 2024 · 15 citations
  5. Safety Alignment Can Be Not Superficial With Explicit Safety SignalsJianwei Li, Jung-Eun KimICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines