ACL Findings2026
From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning
Xiaoyu Xu, Minxin Du, Zitong Li, Zi Liang, Zhibiao Guo, Shiyu Zhang, Peizhao Hu, Qingqing Ye, Haibo Hu
Abstract
Although machine unlearning is essential for removing private, harmful, or copyrighted content from LLMs, current benchmarks often fail to faithfully represent the true "forgetting scope" learned by the model. We formalize two distinct unlearning granularities, domain-level and instance-level, and propose BiForget, an automated framework for synthesizing high-quality forget sets. Unlike prior work relying on external generators, BiForget exploits the target model per se to elicit data that matches its internal knowledge distribution through seedguided and adversarial prompting. Our experiments across diverse benchmarks show that it achieves a superior balance of relevance, diversity, and efficiency. Quantitatively, in the Harry Potter domain, it improves relevance by ∼20 and diversity by ∼0.05 while halving the total data size compared to SOTAs. Ultimately, it facilitates more robust forgetting and better utility preservation, providing a more rigorous foundation for evaluating LLM unlearning. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Large Language Model Unlearning via Embedding-Corrupted Prompts cites Measuring Massive Multitask Language Understanding
- Large Language Model Unlearning via Embedding-Corrupted Prompts cites In-Context Unlearning: Language Models as Few-Shot Unlearners
- Large Language Model Unlearning via Embedding-Corrupted Prompts cites Detecting Pretraining Data from Large Language Models
- Detecting Pretraining Data from Large Language Models cites SimCSE: Simple Contrastive Learning of Sentence Embeddings
Builds on18
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
- Detecting Pretraining Data from Large Language ModelsWeijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang et al.ICLR 2024 · 365 citations
- In-Context Unlearning: Language Models as Few-Shot UnlearnersMartin Pawelczyk, Seth Neel, Himabindu LakkarajuICML 2024 · 217 citations
- Large Language Model Unlearning via Embedding-Corrupted PromptsChris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, Yang LiuNeurIPS 2024 · 138 citations
Related papers
- Reveal and Release: Iterative LLM Unlearning with Self-generated DataLinxi Xie, Xin Teng, Shichang Ke, Hongyi Wen et al.EMNLP Findings 2025
- OBLIVIATE: Robust and Practical Machine Unlearning for Large Language ModelsXiaoyu Xu, Minxin Du, Qingqing Ye, Haibo HuEMNLP 2025 · 1 citation
- To Forget or Not? Towards Practical Knowledge Unlearning for Large Language ModelsBozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu et al.EMNLP Findings 2024 · 13 citations
- Direct Token Optimization: A Self-Contained Approach to Large Language Model UnlearningHong Kyu Lee, Ruixuan Liu, Li XiongACL Findings 2026
- Adversarial Mixup UnlearningZhuoyi Peng, Yixuan Tang, Yi YangICLR 2025
