Lune

EMNLP Findings2023

Language-Agnostic Bias Detection in Language Models with Bias Probing

Abdullatif Köksal, Omer Faruk Yalcin, Ahmet Akbiyik, M. Tahir Kilavuz, Anna Korhonen, Hinrich Schütze

Abstract

Pretrained language models (PLMs) are key components in NLP, but they contain strong social biases. Quantifying these biases is challenging because current methods focusing on fill-the-mask objectives are sensitive to slight changes in input. To address this, we propose a bias probing technique called LABDet, for evaluating social bias in PLMs with a robust and language-agnostic method. For nationality as a case study, we show that LABDet "surfaces" nationality bias by training a classifier on top of a frozen PLM on non-nationality sentiment detection. We find consistent patterns of nationality bias across monolingual PLMs in six languages that align with historical and political context. We also show for English BERT that bias surfaced by LABDet correlates well with bias in the pretraining data; thus, our work is one of the few studies that directly links pretraining data to PLM behavior. Finally, we verify LABDet's reliability and applicability to different templates and languages through an extensive set of robustness checks. We publicly share our code and dataset in https://github.com/akoksal/LABDet.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English cites CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
  • Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP cites French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English
  • Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP cites CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLPPieter Delobelle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett et al.EMNLP 2024 · 4 citations

Builds on9

  1. CamemBERT: a Tasty French Language ModelLouis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont et al.ACL 2020 · 703 citations
  2. French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than EnglishAurélie Névéol, Yoann Dupont, Julien Bezançon, Karën FortACL 2022 · 61 citations
  3. How much pretraining data do language models need to learn syntax?Laura Pérez-Mayos, Miguel Ballesteros, Leo WannerEMNLP 2021 · 31 citations
  4. Towards Tracing Knowledge in Language Models Back to the Training DataEkin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong et al.EMNLP Findings 2022 · 30 citations
  5. CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language ModelsNikita Nangia, Clara Vania, Rasika Bhalerao, Samuel R. BowmanEMNLP 2020 · 19 citations

Related papers

  1. The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Language ModelsYan Liu, Yu Liu, Xiaokang Chen, Pin-Yu Chen et al.ICLR 2024 · 32 citations
  2. BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text GenerationTianxiang Sun, Junliang He, Xipeng Qiu, Xuanjing HuangEMNLP 2022 · 22 citations
  3. Probing Toxic Content in Large Pre-Trained Language ModelsNedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song et al.ACL 2021
  4. A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language ModelsYi Zhou, José Camacho-Collados, Danushka BollegalaEMNLP 2023 · 1 citation
  5. Auto-Debias: Debiasing Masked Language Models with Automated Biased PromptsYue Guo, Yi Yang, Ahmed AbbasiACL 2022
Dusk over the sea between two cliffs drawn in fine vertical lines