On the Misalignment Between Data Learnability and Forgettability in Machine Unlearning
Zijie Pan, Zuobin Ying, Yajie Wang, Wanlei Zhou
摘要
We report a structural mismatch between a data point's learnability, which measures how quickly it improves the loss, and its forgettability, which captures how strongly it anchors the final model parameters, an aspect ignored by prior machine unlearning frameworks such as SISA, Fisher Forget, and influence based fine tuning. To make this gap measurable we introduce Unlearning Gradient Sensitivity (UGS), an influence score computable with a single Hutch++ sketch, and derive the Learnability-Forgettability Divergence (LFD), the Jensen-Shannon distance between the model's learning and forgetting distributions. We prove that UGS dispersion decays exponentially only under explicit regularisation and that LFD converges to zero when its weight grows sub-linearly relative to the UGS term. Building on these findings, we introduce Dual-Aware Training (DAT)-a lightweight regularization method that reduces variability in how easily data points can be forgotten and aligns learning and forgetting behaviors during training. On CIFAR-10, MNIST, and IMDB, DAT maintains the original model accuracy while cutting forgettability divergence in half and significantly lowering the cost of certified unlearning, showing that it's effective to make models forgettable from the start.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper11
- Machine UnlearningLucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia 等S&P 2021 · 被引用 1,381 次
- What Neural Networks Memorize and Why: Discovering the Long Tail via Influence EstimationVitaly Feldman, Chiyuan ZhangNeurIPS 2020 · 被引用 674 次
- Adaptive Machine UnlearningVarun Gupta, Christopher Jung, Seth Neel, Aaron Roth 等NeurIPS 2021 · 被引用 262 次
- Variational Bayesian UnlearningQuoc Phong Nguyen, Bryan Kian Hsiang Low, Patrick JailletNeurIPS 2020 · 被引用 198 次
- Exploring Example Influence in Continual LearningQing Sun, Fan Lyu, Fanhua Shang, Wei Feng 等NeurIPS 2022 · 被引用 69 次
相关 Paper
- MUter: Machine Unlearning on Adversarially Trained ModelsJunxu Liu, Mingsheng Xue, Jian Lou, Xiaoyu Zhang 等ICCV 2023 · 被引用 36 次
- Efficient Verified Unlearning For DistillationYijun Quan, Zushu Li, Giovanni MontanaNeurIPS 2025 · 被引用 1 次
- LoTUS: Large-Scale Machine Unlearning with a Taste of UncertaintyChristoforos N. Spartalis, Theodoros Semertzidis, Efstratios Gavves, Petros DarasCVPR 2025
- Tackling Fake Forgetting through Uncertainty QuantificationYingdan Shi, Sijia Liu, Kaize Ding, Ren WangICML 2026 · 被引用 1 次
- Remaining-data-free Machine Unlearning by Suppressing Sample ContributionXinwen Cheng, Zhehao Huang, Wenxing Zhou, Zhengbao He 等ICLR 2026 · 被引用 11 次
