Lune

ACL Findings2025

Harnessing Whisper for Prosodic Stress Analysis

Samuel S. Sohn, Sten Knutsen, Karin Stromswold

Abstract

Prosody affects how people produce and understand language, yet studies of how it does so have been hindered by the lack of efficient tools for analyzing prosodic stress. We fine-tune OpenAI Whisper large-v2, a stateof-the-art speech recognition model, to recognize phrasal, lexical, and contrastive stress using a small, carefully annotated dataset. Our results show that Whisper can learn distinct, gender-specific stress patterns to achieve nearhuman and super-human accuracy in stress classification and transfer its learning from one type of stress to another, surpassing traditional machine learning models. Furthermore, we explore how acoustic context influences its performance and propose a novel black-box evaluation method for characterizing the decision boundaries used by Whisper for prosodic stress interpretation. These findings open new avenues for large-scale, automated prosody research. Models can be found at github.com/SSSohn/ProsodyBench. Stress Minimal Pair Transcription Phrasal The <greenhouse / green house> spoils the view. Phrasal There's a <darkroom / dark room> in this house. Phrasal The <whiteboard / white board> needs cleaning. Phrasal That <hotdog / hot dog> is under the table. Phrasal A <blackbird / black bird> just flew past. Phrasal His <wetsuit / wet suit> is on the floor. Phrasal That <bluebell / blue bell> is pretty. Phrasal The <bullseye / bull's eye> is red. Lexical <DIFfer / deFER> Lexical <DIScard / disCARD> Lexical <DIScount / disCOUNT> Lexical <INcrease / inCREASE> Lexical <INdent / inDENT> Lexical <INsert / inSERT> Lexical <INsight / inCITE> Lexical <INsult / inSULT> Contra. The <BLACK cow / black COW> has the ball. Contra. The <BLACK sheep / black SHEEP> has the ball. Contra. The <BLUE cow / blue COW> has the ball. Contra. The <BLUE sheep / blue SHEEP> has the ball. Contra. The <RED cow / red COW> has the ball. Contra. The <RED sheep / red SHEEP> has the ball. Contra. The <WHITE cow / white COW> has the ball. Contra. The <WHITE sheep / white SHEEP> has the ball.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on1

  1. Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations

Related papers

  1. Fairness in Automatic Speech Recognition Isn't a One-Size-Fits-AllHend Elghazaly, Bahman Mirheidari, Heidi Christensen, Nafise Sadat MoosaviEMNLP Findings 2025
  2. StressTest: Can YOUR Speech LM Handle the Stress?Iddo Yosha, Gallil Maimon, Yossi AdiACL Findings 2026 · 1 citation
  3. Beyond WER: Probing Whisper's Sub-token Decoder Across Diverse Language Resource LevelsSiyu Liang, Nicolas Ballier, Gina-Anne Levow, Richard A. WrightEMNLP 2025
  4. Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance GapsGiuseppe Attanasio, Beatrice Savoldi, Dennis Fucci, Dirk HovyEMNLP 2024 · 5 citations
  5. Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation ModelsYuchen Hu, Chen Chen, Chao-Han Huck Yang, Chengwei Qin et al.NeurIPS 2024 · 14 citations
Dusk over the sea between two cliffs drawn in fine vertical lines