Lune

EMNLP Findings2025

Fairness in Automatic Speech Recognition Isn't a One-Size-Fits-All

Hend Elghazaly, Bahman Mirheidari, Heidi Christensen, Nafise Sadat Moosavi

Abstract

Modern Automatic Speech Recognition (ASR) systems are increasingly deployed in highstakes settings, including clinical interviews, public services, and educational tools, where equitable performance across speaker groups is essential. While pre-trained speech models like Whisper achieve strong overall accuracy, they often exhibit inconsistent grouplevel performance that varies across domains. These disparities are not fixed properties of the model, but emerge from the interaction between model, data, and task-posing challenges for fairness interventions designed indomain. We frame fairness in ASR as a generalisation problem. We fine-tune a Whisper model on the Fair-Speech corpus using four strategies: basic fine-tuning, demographic rebalancing, gender-swapped data augmentation, and a novel contrastive learning objective that encourages gender-invariant representations. We evaluate performance across multiple aspects of fairness and utility, both in-domain and on three out-of-domain test sets: LibriSpeech, EdAcc, and CognoSpeak. Our findings show that the method with the best in-domain fairness performed worst out-of-domain, illustrating that fairness gains do not always generalise. Demographic balancing generalises more consistently, while our contrastive method offers a practical alternative: it achieves stable, crossdomain fairness improvements without requiring changes to the training data distribution, and with minimal accuracy trade-offs.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Robust Speech Recognition via Large-Scale Weak Supervision cites wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
  • FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders cites A Simple Framework for Contrastive Learning of Visual Representations
  • wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations cites A Simple Framework for Contrastive Learning of Visual Representations
  • Understanding the Behaviour of Contrastive Loss cites A Simple Framework for Contrastive Learning of Visual Representations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
  3. Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
  4. FairFil: Contrastive Neural Debiasing Method for Pretrained Text EncodersPengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si et al.ICLR 2021 · 50 citations
  5. Understanding the Behaviour of Contrastive LossFeng Wang, Huaping LiuCVPR 2021

Related papers

  1. Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition modelsFelix Herron, Solange Rossato, Alexandre Allauzen, François PortetACL Findings 2026
  2. "It feels like we're not meeting the criteria": Examining and Mitigating the Cascading Effects of Bias in Automatic Speech Recognition in Spoken Language InterfacesKelechi Ezema, Chelsea Chandler, Rosy Southwell, Niranjan Cholendiran et al.CHI 2025 · 8 citations
  3. Learning Fair Representation via Distributional Contrastive DisentanglementChangdae Oh, Heeji Won, Junhyuk So, Taero Kim et al.KDD 2022 · 29 citations
  4. Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech RecognitionYi-Cheng Lin, Yu-Hsuan Li Liang, Hsuan Su, Tzu-Quan Lin et al.ACL Findings 2026
  5. Fairness Beyond Performance: Revealing Reliability Disparities Across Groups in Legal NLPT. Y. S. S. Santosh, Irtiza ChowdhuryACL 2025
Dusk over the sea between two cliffs drawn in fine vertical lines