Lune

CHI2023Top-tier venue

Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

Zahra Ashktorab, Benjamin Hoover, Mayank Agarwal, Casey Dugan, Werner Geyer, Hao Bang Yang, Mikhail Yurochkin

Abstract

Mitigating algorithmic bias is a critical task in the development and deployment of machine learning models. While several toolkits exist to aid machine learning practitioners in addressing fairness issues, little is known about the strategies practitioners employ to evaluate model fairness and what factors influence their assessment, particularly in the context of text classification. Two common approaches of evaluating the fairness of a model are group fairness and individual fairness. We run a study with Machine Learning practitioners (n=24) to understand the strategies used to evaluate models. Metrics presented to practitioners (group vs. individual fairness) impact which models they consider fair. Participants focused on risks associated with underpredicting / overpredicting and model sensitivity relative to identity token manipulations. We discover fairness assessment strategies involving personal experiences or how users form groups of identity tokens to test model fairness. We provide recommendations for interactive tools for evaluating fairness in text classification.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
  • "I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment cites Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
  • "I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment cites EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders
  • Towards a Non-Ideal Methodological Framework for Responsible ML cites Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
  • Towards a Non-Ideal Methodological Framework for Responsible ML cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
  • EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders cites Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
  • Post-processing for Individual Fairness cites Training individually fair ML models with sensitive subspace robustness
  • Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams cites Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Towards a Non-Ideal Methodological Framework for Responsible MLRamaravind Kommiya Mothilal, Shion Guha, Syed Ishtiaque AhmedCHI 2024 · 7 citations
  2. Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry TeamsVanessa Aisyahsari Hanschke, Dylan Rees, Merve Alanyali, David Hopkinson et al.CHI 2024 · 7 citations
  3. EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among StakeholdersLin Luo, Yuri Nakao, Mathieu Chollet, Hiroya Inakoshi et al.CSCW 2025 · 4 citations
  4. "I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness AssessmentLin Luo, Yuri Nakao, Mathieu Chollet, Hiroya Inakoshi et al.CHI 2026 · 1 citation

Builds on14

  1. WILDS: A Benchmark of in-the-Wild Distribution ShiftsPang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie et al.ICML 2021 · 1,773 citations
  2. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AIMichael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, Hanna M. WallachCHI 2020 · 428 citations
  3. Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for SupportMichael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan et al.CSCW 2022 · 149 citations
  4. Training individually fair ML models with sensitive subspace robustnessMikhail Yurochkin, Amanda Bower, Yuekai SunICLR 2020 · 123 citations
  5. Post-processing for Individual FairnessFelix Petersen, Debarghya Mukherjee, Yuekai Sun, Mikhail YurochkinNeurIPS 2021 · 115 citations

Related papers

  1. The Landscape and Gaps in Open Source Fairness ToolkitsMichelle Seng Ah Lee, Jatinder SinghCHI 2021 · 117 citations
  2. Two Simple Ways to Learn Individual Fairness Metrics from DataDebarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, Yuekai SunICML 2020 · 109 citations
  3. Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML ToolkitsBrianna Richardson, Jean Garcia-Gathright, Samuel F. Way, Jennifer Thom et al.CHI 2021 · 56 citations
  4. OxonFair: A Flexible Toolkit for Algorithmic FairnessEoin Delaney, Zihao Fu, Sandra Wachter, Brent D. Mittelstadt et al.NeurIPS 2024 · 13 citations
  5. Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairnessSumon Biswas, Hridesh RajanFSE 2020 · 96 citations
Dusk over the sea between two cliffs drawn in fine vertical lines