Fast Model DeBias with Machine Unlearning
Ruizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai, Tianxiang Hu, Jin Hao, Yang Feng, Joey Tianyi Zhou, Jian Wu, Zuozhu Liu
Abstract
Recent discoveries have revealed that deep neural networks might behave in a biased manner in many real-world scenarios. For instance, deep networks trained on a large-scale face recognition dataset CelebA tend to predict blonde hair for females and black hair for males. Such biases not only jeopardize the robustness of models but also perpetuate and amplify social biases, which is especially concerning for automated decision-making processes in healthcare, recruitment, etc., as they could exacerbate unfair economic and social inequalities among different groups. Existing debiasing methods suffer from high costs in bias labeling or model re-training, while also exhibiting a deficiency in terms of elucidating the origins of biases within the model. To this respect, we propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases inherent in trained models. The FMD identifies biased attributes through an explicit counterfactual concept and quantifies the influence of data samples with influence functions. Moreover, we design a machine unlearning-based strategy to efficiently and effectively remove the bias in a trained model with a small counterfactual dataset. Experiments on the Colored MNIST, CelebA, and Adult Income datasets along with experiments with large language models demonstrate that our method achieves superior or competing accuracies compared with state-of-the-art methods while attaining significantly fewer biases and requiring much less debiasing cost. Notably, our method requires only a small external dataset and updating a minimal amount of model parameters, without the requirement of access to training data that may be too large or unavailable in practice.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
20 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Certified Data Removal from Machine Learning Models cites Machine Unlearning
- Machine Unlearning cites Certified Data Removal from Machine Learning Models
- Boosting Alignment for Post-Unlearning Text-to-Image Generative Models cites Certified Data Removal from Machine Learning Models
- Boosting Alignment for Post-Unlearning Text-to-Image Generative Models cites Machine Unlearning
- Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement cites Certified Data Removal from Machine Learning Models
- Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement cites Machine Unlearning
- Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity cites Certified Data Removal from Machine Learning Models
- Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity cites Machine Unlearning
- Learning De-biased Representations with Biased Representations cites Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
Cited by top-tier papers20
- Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language ModelsJiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi et al.NeurIPS 2024 · 62 citations
- Unified Gradient-Based Machine Unlearning with Remain Geometry EnhancementZhehao Huang, Xinwen Cheng, JingHao Zheng, Haoran Wang et al.NeurIPS 2024 · 57 citations
- Ferrari: Federated Feature Unlearning via Optimizing Feature SensitivityHanlin Gu, WinKent Ong, Chee Seng Chan, Lixin FanNeurIPS 2024 · 29 citations
- Boosting Alignment for Post-Unlearning Text-to-Image Generative ModelsMyeongseob Ko, Henry Li, Zhun Wang, Jonathan Patsenker et al.NeurIPS 2024 · 22 citations
- Learning Counterfactual Outcomes Under Rank PreservationPeng Wu, Haoxuan Li, Chunyuan Zheng, Yan Zeng et al.NeurIPS 2025 · 7 citations
Builds on22
- Machine UnlearningLucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia et al.S&P 2021 · 1,381 citations
- Certified Data Removal from Machine Learning ModelsChuan Guo, Tom Goldstein, Awni Y. Hannun, Laurens van der MaatenICML 2020 · 633 citations
- Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image RepresentationsTianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang et al.ICCV 2019 · 469 citations
- An Investigation of Why Overparameterization Exacerbates Spurious CorrelationsShiori Sagawa, Aditi Raghunathan, Pang Wei Koh, Percy LiangICML 2020 · 436 citations
- Learning De-biased Representations with Biased RepresentationsHyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo et al.ICML 2020 · 332 citations
Related papers
- Data-Centric Explainable Debiasing for Improving Fairness in Pre-trained Language ModelsYingji Li, Mengnan Du, Rui Song, Xin Wang et al.ACL Findings 2024
- Interpretable Debiasing of Vision-Language Models for Social FairnessNa Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma et al.CVPR 2026 · 7 citations
- Social Debiasing for Fair Multi-Modal LLMsHarry Cheng, Yangyang Guo, Qing Guo, Ming-Hsuan Yang et al.ICCV 2025 · 1 citation
- Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying PromptsYujie Lin, Kunquan Li, Yixuan Liao, Xiaoxin Chen et al.ICLR 2026 · 6 citations
- Knowing Bias, Doing Better: Mitigating Social Bias in LLMs via Know-Bias Neuron EnhancementJinhao Pan, Chahat Raj, Anjishnu Mukherjee, Sina Mansouri et al.ICML 2026
