Lune

ACL Findings2021

Is the Lottery Fair? Evaluating Winning Tickets Across Demographics

Victor Petrén Bach Hansen, Anders Søgaard

Abstract

Recent studies have suggested that weight pruning, e.g. using lottery ticket extraction techniques (Frankle and Carbin, 2018), comes at the risk of compromising the group fairness of machine learning models (Paganini, 2020;Hooker et al., 2020), but to the best of our knowledge, no one has empirically evaluated this hypothesis at scale in the context of natural language processing.We present experiments with two text classification datasets annotated with demographic information: the Trustpilot Corpus (sentiment) and CivilComments (toxicity).We evaluate the fairness of lottery ticket extraction through layer-wise and global weight pruning across three languages and two tasks.Our results suggest that there is a small increase in group disparity, which is most pronounced at high pruning rates and correlates with instability.The fairness of models trained with distributionally robust optimization objectives is sometimes less sensitive to pruning, but results are not consistent.The code for our experiments is available at https://github. com/vpetren/fairness_lottery.0 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: CivilComments, FFNN, layerwise Group Disparity, FSP=0.4970 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 F1-score Pruning scores: CivilComments, FFNN, global Group Disparity, FSP=0.3390 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 F1-score Pruning scores: Trustpilot-DK, FFNN, layerwiseGroup Disparity, FSP=-0.1830 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-DK, FFNN, globalGroup Disparity, FSP=0.2270 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-DE, FFNN, layerwiseGroup Disparity, FSP=0.2810 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 F1-score Pruning scores: Trustpilot-DE, FFNN, globalGroup Disparity, FSP=1.375 0 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-UK, FFNN, layerwiseGroup Disparity, FSP=-0.2300 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-UK, FFNN, global Group Disparity, FSP=1.0540 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: CivilComments, FFNN, layerwise Group Disparity, FSP=0.4970 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 F1-score Pruning scores: CivilComments, FFNN, globalGroup Disparity, FSP=0.3390 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 F1-scorePruning scores: Trustpilot-DK, FFNN, layerwise, DRO Group Disparity, FSP=-0.0440 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-DK, FFNN, global, DRO Group Disparity, FSP=0.3510 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-DE, FFNN, layerwise, DRO Group Disparity, FSP=0.3210 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 % of weights pruned 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-score Pruning scores: Trustpilot-DE, FFNN, global, DRO Group Disparity, FSP=0.875 0 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-scorePruning scores: Trustpilot-UK, FFNN, layerwise, DRO Group Disparity, FSP=0.1430 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 F1-scorePruning scores: Trustpilot-UK, FFNN, global, DRO Group Disparity, FSP=0.0360 .00 0 3 5 .00 0 5 7 .7 5 0 7 2 .5 3 7 8 2 .1 4 9 8 8 .3 9 7 9 2 .4 5 8 9 5 .0 9 8 9 6 .8 1 4 9 7 .9 2 9 9 8 .6 5 4 9 9 .1 2 5 9 9 .4 3 1 9 9 .6 3 0 9 9 .7 6 0 9 9 .8 4 4 9 9 .89 8 9 9 .9 3 4 9 9 .9 5 7 9 9 .97 2 % of weights pruned 0.2 0.3 0.4 0.5 0.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Movement Pruning: Adaptive Sparsity by Fine-Tuning cites Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
  • Movement Pruning: Adaptive Sparsity by Fine-Tuning cites Linear Mode Connectivity and the Lottery Ticket Hypothesis
  • Linear Mode Connectivity and the Lottery Ticket Hypothesis cites Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
  • Fair Scratch Tickets: Finding Fair Sparse Networks without Weight Training cites Linear Mode Connectivity and the Lottery Ticket Hypothesis
  • Fair Scratch Tickets: Finding Fair Sparse Networks without Weight Training cites Large-Scale Methods for Distributionally Robust Optimization
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Fair Scratch Tickets: Finding Fair Sparse Networks without Weight TrainingPengwei Tang, Wei Yao, Zhicong Li, Yong LiuCVPR 2023

Builds on7

  1. Linear Mode Connectivity and the Lottery Ticket HypothesisJonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, Michael CarbinICML 2020 · 750 citations
  2. Movement Pruning: Adaptive Sparsity by Fine-TuningVictor Sanh, Thomas Wolf, Alexander M. RushNeurIPS 2020 · 656 citations
  3. An Investigation of Why Overparameterization Exacerbates Spurious CorrelationsShiori Sagawa, Aditi Raghunathan, Pang Wei Koh, Percy LiangICML 2020 · 436 citations
  4. Large-Scale Methods for Distributionally Robust OptimizationDaniel Levy, Yair Carmon, John C. Duchi, Aaron SidfordNeurIPS 2020 · 281 citations
  5. Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPHaonan Yu, Sergey Edunov, Yuandong Tian, Ari S. MorcosICLR 2020 · 156 citations

Related papers

  1. Efficient DNN-Powered Software with Fair Sparse ModelsXuanqi Gao, Weipeng Jiang, Juan Zhai, Shiqing Ma et al.ISSTA 2024
  2. Preserving Fairness and Safety in Quantized LLMs Through Critical Weight ProtectionMuhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri KotoACL Findings 2026 · 1 citation
  3. BiasWipe: Mitigating Unintended Bias in Text Classifiers through Model InterpretabilityMamta Mamta, Rishikant Chigrupaatii, Asif EkbalEMNLP 2024 · 4 citations
  4. A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language ModelsKrithika Ramesh, Arnav Chavan, Shrey Pandit, Sunayana SitaramACL 2023 · 14 citations
  5. On Fairness of Task Arithmetic: The Role of Task VectorsLaura Gomezjurado Gonzalez, Hiroki Naganuma, Kotaro Yoshida, Takafumi Horie et al.ICLR 2026 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines