EMNLP Findings2024
Emosical: An Emotion-Annotated Musical Theatre Dataset
Hayoon Kim, Ahyeon Choi, Sungho Lee, Hyun Jung, Kyogu Lee
Abstract
This paper presents Emosical, a multimodal open-source dataset of musical films. Emosical comprises video, vocal audio, text, and character identity paired samples with annotated emotion tags. Emosical provides rich emotion annotations for each sample by inferring the background story of the characters. To achieve this, we leverage the musical theatre script, which contains the characters' complete background stories and narrative contexts. The annotation pipeline includes feeding the speaking character, text, global persona, and context of the dialogue and song track into a large language model. To verify the effectiveness of our tagging scheme, we perform an ablation study by bypassing each step of the pipeline. The ablation results show the usefulness of each component in generating accurate emotion tags. A subjective test is conducted to compare the generated tags of each ablation result. We also perform a statistical analysis to find out the global characteristics of the collected emotion tags. Emosical would enable expressive synthesis and tagging of the speech and singing voice in the musical theatre domain in future research. Emosical is publicly available at https://github.com/gillosae/emosical.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on4
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale CorpusRongjie Huang, Feiyang Chen, Yi Ren, Jinglin Liu et al.ACM MM 2021 · 75 citations
- V2C: Visual Voice CloningQi Chen, Mingkui Tan, Yuankai Qi, Jiaqiu Zhou et al.CVPR 2022
Related papers
- VMChill: A Dataset for Fine-Grained Visual-Musical SynergyXiaowei Chi, Zeyue Tian, Jialiang Chen, Wei XueAAAI 2026
- Observe before Generate: Emotion-Cause aware Video Caption for Multimodal Emotion Cause Generation in ConversationsFanfan Wang, Heqing Ma, Xiangqing Shen, Jianfei Yu et al.ACM MM 2024 · 6 citations
- Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face ConversationSe Jin Park, Chae Won Kim, Hyeongseop Rha, Minsu Kim et al.ACL 2024
- Towards Authentic Movie Dubbing with Retrieve-Augmented Director-Actor Interaction LearningRui Liu, Yuan Zhao, Zhenqi JiaAAAI 2026
- How You Feelin'? Learning Emotions and Mental States in Movie ScenesDhruv Srivastava, Aditya Kumar Singh, Makarand TapaswiCVPR 2023
