Lune

EMNLP Findings2021

Don't Discard All the Biased Instances: Investigating a Core Assumption in Dataset Bias Mitigation Techniques

Hossein Amirkhani, Mohammad Taher Pilehvar

Abstract

Existing techniques for mitigating dataset bias often leverage a biased model to identify biased instances. The role of these biased instances is then reduced during the training of the main model to enhance its robustness to out-of-distribution data. A common core assumption of these techniques is that the main model handles biased instances similarly to the biased model, in that it will resort to biases whenever available. In this paper, we show that this assumption does not hold in general. We carry out a critical investigation on two well-known datasets in the domain, MNLI and FEVER, along with two biased instance detection methods, partial-input and limitedcapacity models. Our experiments show that in around a third to a half of instances, the biased model is unable to predict the main model's behavior, highlighted by the significantly different parts of the input on which they base their decisions. Based on a manual validation, we also show that this estimate is highly in line with human interpretation. Our findings suggest that down-weighting of instances detected by bias detection methods, which is a widely-practiced procedure, is an unnecessary waste of training data. We release our code to facilitate reproducibility and future research. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Improving the robustness of NLI models with minimax training cites End-to-End Bias Mitigation by Modelling Biases in Corpora
  • Improving the robustness of NLI models with minimax training cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  • Improving the robustness of NLI models with minimax training cites Learning from others' mistakes: Avoiding dataset biases without modeling them
  • Improving the robustness of NLI models with minimax training cites Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
  • Counterfactual Debiasing for Fact Verification cites End-to-End Bias Mitigation by Modelling Biases in Corpora
  • Counterfactual Debiasing for Fact Verification cites Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
  • Learning from others' mistakes: Avoiding dataset biases without modeling them cites End-to-End Bias Mitigation by Modelling Biases in Corpora
  • Learning from others' mistakes: Avoiding dataset biases without modeling them cites WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  • Interventional Training for Out-Of-Distribution Natural Language Understanding cites End-to-End Bias Mitigation by Modelling Biases in Corpora
  • Interventional Training for Out-Of-Distribution Natural Language Understanding cites Learning from others' mistakes: Avoiding dataset biases without modeling them
  • Interventional Training for Out-Of-Distribution Natural Language Understanding cites Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
  • Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles cites Adversarial Filters of Dataset Biases
  • Mitigating Shortcut Learning with InterpoLated Learning cites End-to-End Bias Mitigation by Modelling Biases in Corpora
  • Mitigating Shortcut Learning with InterpoLated Learning cites Improving the robustness of NLI models with minimax training
  • Mitigating Shortcut Learning with InterpoLated Learning cites Learning from others' mistakes: Avoiding dataset biases without modeling them
  • Mitigating Shortcut Learning with InterpoLated Learning cites Adversarial Filters of Dataset Biases
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Counterfactual Debiasing for Fact VerificationWeizhi Xu, Qiang Liu, Shu Wu, Liang WangACL 2023 · 26 citations
  2. Improving the robustness of NLI models with minimax trainingMichalis Korakakis, Andreas VlachosACL 2023 · 4 citations
  3. Interventional Training for Out-Of-Distribution Natural Language UnderstandingSicheng Yu, Jing Jiang, Hao Zhang, Yulei Niu et al.EMNLP 2022 · 3 citations
  4. Mitigating Shortcut Learning with InterpoLated LearningMichalis Korakakis, Andreas Vlachos, Adrian WellerACL 2025

Builds on6

  1. WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
  2. Adversarial Filters of Dataset BiasesRonan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers et al.ICML 2020 · 242 citations
  3. End-to-End Bias Mitigation by Modelling Biases in CorporaRabeeh Karimi Mahabadi, Yonatan Belinkov, James HendersonACL 2020 · 136 citations
  4. Learning from others' mistakes: Avoiding dataset biases without modeling themVictor Sanh, Thomas Wolf, Yonatan Belinkov, Alexander M. RushICLR 2021 · 123 citations
  5. Learning to Model and Ignore Dataset Bias with Mixed Capacity EnsemblesChristopher Clark, Mark Yatskar, Luke ZettlemoyerEMNLP Findings 2020 · 48 citations

Related papers

  1. Investigating Ensemble Methods for Model Robustness Improvement of Text ClassifiersJieyu Zhao, Xuezhi Wang, Yao Qin, Jilin Chen et al.EMNLP Findings 2022
  2. Improving Bias Mitigation through Bias Experts in Natural Language UnderstandingEojin Jeon, Mingyu Lee, Juhyeong Park, Yeachan Kim et al.EMNLP 2023 · 3 citations
  3. Towards Robustifying NLI Models Against Lexical Dataset BiasesXiang Zhou, Mohit BansalACL 2020 · 36 citations
  4. Towards Debiasing NLU Models from Unknown BiasesPrasetya Ajie Utama, Nafise Sadat Moosavi, Iryna GurevychEMNLP 2020 · 3 citations
  5. Debiasing Methods in Natural Language Understanding Make Bias More AccessibleMichael Mendelson, Yonatan BelinkovEMNLP 2021 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines