ACL Findings2025
Harnessing Whisper for Prosodic Stress Analysis
Samuel S. Sohn, Sten Knutsen, Karin Stromswold
Abstract
Prosody affects how people produce and understand language, yet studies of how it does so have been hindered by the lack of efficient tools for analyzing prosodic stress. We fine-tune OpenAI Whisper large-v2, a stateof-the-art speech recognition model, to recognize phrasal, lexical, and contrastive stress using a small, carefully annotated dataset. Our results show that Whisper can learn distinct, gender-specific stress patterns to achieve nearhuman and super-human accuracy in stress classification and transfer its learning from one type of stress to another, surpassing traditional machine learning models. Furthermore, we explore how acoustic context influences its performance and propose a novel black-box evaluation method for characterizing the decision boundaries used by Whisper for prosodic stress interpretation. These findings open new avenues for large-scale, automated prosody research. Models can be found at github.com/SSSohn/ProsodyBench. Stress Minimal Pair Transcription Phrasal The <greenhouse / green house> spoils the view. Phrasal There's a <darkroom / dark room> in this house. Phrasal The <whiteboard / white board> needs cleaning. Phrasal That <hotdog / hot dog> is under the table. Phrasal A <blackbird / black bird> just flew past. Phrasal His <wetsuit / wet suit> is on the floor. Phrasal That <bluebell / blue bell> is pretty. Phrasal The <bullseye / bull's eye> is red. Lexical <DIFfer / deFER> Lexical <DIScard / disCARD> Lexical <DIScount / disCOUNT> Lexical <INcrease / inCREASE> Lexical <INdent / inDENT> Lexical <INsert / inSERT> Lexical <INsight / inCITE> Lexical <INsult / inSULT> Contra. The <BLACK cow / black COW> has the ball. Contra. The <BLACK sheep / black SHEEP> has the ball. Contra. The <BLUE cow / blue COW> has the ball. Contra. The <BLUE sheep / blue SHEEP> has the ball. Contra. The <RED cow / red COW> has the ball. Contra. The <RED sheep / red SHEEP> has the ball. Contra. The <WHITE cow / white COW> has the ball. Contra. The <WHITE sheep / white SHEEP> has the ball.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on1
Related papers
- Fairness in Automatic Speech Recognition Isn't a One-Size-Fits-AllHend Elghazaly, Bahman Mirheidari, Heidi Christensen, Nafise Sadat MoosaviEMNLP Findings 2025
- StressTest: Can YOUR Speech LM Handle the Stress?Iddo Yosha, Gallil Maimon, Yossi AdiACL Findings 2026 · 1 citation
- Beyond WER: Probing Whisper's Sub-token Decoder Across Diverse Language Resource LevelsSiyu Liang, Nicolas Ballier, Gina-Anne Levow, Richard A. WrightEMNLP 2025
- Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance GapsGiuseppe Attanasio, Beatrice Savoldi, Dennis Fucci, Dirk HovyEMNLP 2024 · 5 citations
- Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation ModelsYuchen Hu, Chen Chen, Chao-Han Huck Yang, Chengwei Qin et al.NeurIPS 2024 · 14 citations
