EMNLP Findings2025
Fairness in Automatic Speech Recognition Isn't a One-Size-Fits-All
Hend Elghazaly, Bahman Mirheidari, Heidi Christensen, Nafise Sadat Moosavi
Abstract
Modern Automatic Speech Recognition (ASR) systems are increasingly deployed in highstakes settings, including clinical interviews, public services, and educational tools, where equitable performance across speaker groups is essential. While pre-trained speech models like Whisper achieve strong overall accuracy, they often exhibit inconsistent grouplevel performance that varies across domains. These disparities are not fixed properties of the model, but emerge from the interaction between model, data, and task-posing challenges for fairness interventions designed indomain. We frame fairness in ASR as a generalisation problem. We fine-tune a Whisper model on the Fair-Speech corpus using four strategies: basic fine-tuning, demographic rebalancing, gender-swapped data augmentation, and a novel contrastive learning objective that encourages gender-invariant representations. We evaluate performance across multiple aspects of fairness and utility, both in-domain and on three out-of-domain test sets: LibriSpeech, EdAcc, and CognoSpeak. Our findings show that the method with the best in-domain fairness performed worst out-of-domain, illustrating that fairness gains do not always generalise. Demographic balancing generalises more consistently, while our contrastive method offers a practical alternative: it achieves stable, crossdomain fairness improvements without requiring changes to the training data distribution, and with minimal accuracy trade-offs.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Robust Speech Recognition via Large-Scale Weak Supervision cites wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
- FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders cites A Simple Framework for Contrastive Learning of Visual Representations
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations cites A Simple Framework for Contrastive Learning of Visual Representations
- Understanding the Behaviour of Contrastive Loss cites A Simple Framework for Contrastive Learning of Visual Representations
Builds on5
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech RepresentationsAlexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, Michael AuliNeurIPS 2020 · 9,451 citations
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
- FairFil: Contrastive Neural Debiasing Method for Pretrained Text EncodersPengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si et al.ICLR 2021 · 50 citations
- Understanding the Behaviour of Contrastive LossFeng Wang, Huaping LiuCVPR 2021
Related papers
- Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition modelsFelix Herron, Solange Rossato, Alexandre Allauzen, François PortetACL Findings 2026
- "It feels like we're not meeting the criteria": Examining and Mitigating the Cascading Effects of Bias in Automatic Speech Recognition in Spoken Language InterfacesKelechi Ezema, Chelsea Chandler, Rosy Southwell, Niranjan Cholendiran et al.CHI 2025 · 8 citations
- Learning Fair Representation via Distributional Contrastive DisentanglementChangdae Oh, Heeji Won, Junhyuk So, Taero Kim et al.KDD 2022 · 29 citations
- Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech RecognitionYi-Cheng Lin, Yu-Hsuan Li Liang, Hsuan Su, Tzu-Quan Lin et al.ACL Findings 2026
- Fairness Beyond Performance: Revealing Reliability Disparities Across Groups in Legal NLPT. Y. S. S. Santosh, Irtiza ChowdhuryACL 2025
