ACL Findings2024
MELD-ST: An Emotion-aware Speech Translation Dataset
Sirou Chen, Sakiko Yahata, Shuichiro Shimizu, Zhengdong Yang, Yihang Li, Chenhui Chu, Sadao Kurohashi
Abstract
Emotion plays a crucial role in human conversation. This paper underscores the significance of considering emotion in speech translation. We present the MELD-ST dataset for the emotion-aware speech translation task, comprising English-to-Japanese and Englishto-German language pairs. Each language pair includes about 10, 000 utterances annotated with emotion labels from the MELD dataset.
Baseline experiments using the SEAMLESSM4T model on the dataset indicate that fine-tuning with emotion labels can enhance translation performance in some settings, highlighting the need for further research in emotion-aware speech translation systems.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Lune answers from
the full paper · 1 top-tier paper that cites it · related work
Free to start. No credit card required.
Cited by top-tier papers1
Builds on5
- Robust Speech Recognition via Large-Scale Weak SupervisionAlec Radford, Jong Wook Kim, Tao Xu, Greg Brockman et al.ICML 2023 · 6,966 citations
- Direct Speech-to-Speech Translation With Discrete UnitsAnn Lee, Peng-Jen Chen, Changhan Wang, Jiatao Gu et al.ACL 2022 · 235 citations
- BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
- Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE CorpusLuisa Bentivogli, Beatrice Savoldi, Matteo Negri, Mattia Antonino Di Gangi et al.ACL 2020 · 40 citations
- Towards Speech Dialogue Translation Mediating Speakers of Different LanguagesShuichiro Shimizu, Chenhui Chu, Sheng Li, Sadao KurohashiACL Findings 2023 · 2 citations
Related papers
- Generative Error Correction for Emotion-aware Speech-to-text TranslationZhengdong Yang, Sheng Li, Chenhui ChuACL Findings 2025
- MSCTD: A Multimodal Sentiment Chat Translation DatasetYunlong Liang, Fandong Meng, Jinan Xu, Yufeng Chen et al.ACL 2022 · 27 citations
- Aligning Speech Segments Beyond Pure SemanticsKevin Heffernan, Artyom Kozhevnikov, Loïc Barrault, Alexandre Mourachko et al.ACL Findings 2024 · 2 citations
- Learning and Evaluating Emotion Lexicons for 91 LanguagesSven Buechel, Susanna Rücker, Udo HahnACL 2020 · 2 citations
- EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text PromptingGuanrou Yang, Chen Yang, Qian Chen, Ziyang Ma et al.ACM MM 2025 · 16 citations
