On the Misalignment Between Data Learnability and Forgettability in Machine Unlearning
Zijie Pan, Zuobin Ying, Yajie Wang, Wanlei Zhou
Abstract
We report a structural mismatch between a data point's learnability, which measures how quickly it improves the loss, and its forgettability, which captures how strongly it anchors the final model parameters, an aspect ignored by prior machine unlearning frameworks such as SISA, Fisher Forget, and influence based fine tuning. To make this gap measurable we introduce Unlearning Gradient Sensitivity (UGS), an influence score computable with a single Hutch++ sketch, and derive the Learnability-Forgettability Divergence (LFD), the Jensen-Shannon distance between the model's learning and forgetting distributions. We prove that UGS dispersion decays exponentially only under explicit regularisation and that LFD converges to zero when its weight grows sub-linearly relative to the UGS term. Building on these findings, we introduce Dual-Aware Training (DAT)-a lightweight regularization method that reduces variability in how easily data points can be forgotten and aligns learning and forgetting behaviors during training. On CIFAR-10, MNIST, and IMDB, DAT maintains the original model accuracy while cutting forgettability divergence in half and significantly lowering the cost of certified unlearning, showing that it's effective to make models forgettable from the start.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on11
- Machine UnlearningLucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia et al.S&P 2021 · 1,381 citations
- What Neural Networks Memorize and Why: Discovering the Long Tail via Influence EstimationVitaly Feldman, Chiyuan ZhangNeurIPS 2020 · 674 citations
- Adaptive Machine UnlearningVarun Gupta, Christopher Jung, Seth Neel, Aaron Roth et al.NeurIPS 2021 · 262 citations
- Variational Bayesian UnlearningQuoc Phong Nguyen, Bryan Kian Hsiang Low, Patrick JailletNeurIPS 2020 · 198 citations
- Exploring Example Influence in Continual LearningQing Sun, Fan Lyu, Fanhua Shang, Wei Feng et al.NeurIPS 2022 · 69 citations
Related papers
- MUter: Machine Unlearning on Adversarially Trained ModelsJunxu Liu, Mingsheng Xue, Jian Lou, Xiaoyu Zhang et al.ICCV 2023 · 36 citations
- Efficient Verified Unlearning For DistillationYijun Quan, Zushu Li, Giovanni MontanaNeurIPS 2025 · 1 citation
- LoTUS: Large-Scale Machine Unlearning with a Taste of UncertaintyChristoforos N. Spartalis, Theodoros Semertzidis, Efstratios Gavves, Petros DarasCVPR 2025
- Tackling Fake Forgetting through Uncertainty QuantificationYingdan Shi, Sijia Liu, Kaize Ding, Ren WangICML 2026 · 1 citation
- Remaining-data-free Machine Unlearning by Suppressing Sample ContributionXinwen Cheng, Zhehao Huang, Wenxing Zhou, Zhengbao He et al.ICLR 2026 · 11 citations
