EMNLP Findings2024
BadFair: Backdoored Fairness Attacks with Group-conditioned Triggers
Jiaqi Xue, Qian Lou, Mengxin Zheng
Abstract
Although many works have been developed to improve the fairness of deep learning models, their resilience against malicious attacks-particularly the growing threat of backdoor attacks-has not been thoroughly explored. Attacking fairness is crucial because compromised models can introduce biased outcomes, undermining trust and amplifying inequalities in sensitive applications like hiring, healthcare, and law enforcement. This highlights the urgent need to understand how fairness mechanisms can be exploited and to develop defenses that ensure both fairness and robustness. We introduce BadFair, a novel backdoored fairness attack methodology. BadFair stealthily crafts a model that operates with accuracy and fairness under regular conditions but, when activated by certain triggers, discriminates and produces incorrect results for specific groups. This type of attack is particularly stealthy and dangerous, as it circumvents existing fairness detection methods, maintaining an appearance of fairness in normal use. Our findings reveal that BadFair achieves a more than 85% attack success rate in attacks aimed at target groups on average while only incurring a minimal accuracy loss. Moreover, it consistently exhibits a significant discrimination score, distinguishing between pre-defined target and non-target attacked groups across various datasets and models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on14
- Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
- Poisoning Language Models During Instruction TuningAlexander Wan, Eric Wallace, Sheng Shen, Dan KleinICML 2023 · 319 citations
- Weight Poisoning Attacks on Pretrained ModelsKeita Kurita, Paul Michel, Graham NeubigACL 2020 · 312 citations
- Piccolo: Exposing Complex Backdoors in NLP Transformer ModelsYingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An et al.S&P 2022 · 100 citations
- Exacerbating Algorithmic Bias through Fairness AttacksNinareh Mehrabi, Muhammad Naveed, Fred Morstatter, Aram GalstyanAAAI 2021 · 76 citations
Related papers
- PatchBackdoor: Backdoor Attack against Deep Neural Networks without Model ModificationYizhen Yuan, Rui Kong, Shenghao Xie, Yuanchun Li et al.ACM MM 2023 · 12 citations
- DEFEAT: Deep Hidden Feature Backdoor Attacks by Imperceptible Perturbation and Latent Representation ConstraintsZhendong Zhao, Xiaojun Chen, Yuexin Xuan, Ye Dong et al.CVPR 2022 · 72 citations
- Backdoor Attacks Against Deep Learning Systems in the Physical WorldEmily Wenger, Josephine Passananti, Arjun Nitin Bhagoji, Yuanshun Yao et al.CVPR 2021
- DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited DataDorde Popovic, Amin Sadeghi, Ting Yu, Sanjay Chawla et al.USENIX Security 2025
- Black-box Detection of Backdoor Attacks with Limited Information and DataYinpeng Dong, Xiao Yang, Zhijie Deng, Tianyu Pang et al.ICCV 2021 · 128 citations
