EMNLP Findings2021
DialogueTRM: Exploring Multi-Modal Emotional Dynamics in a Conversation
Yuzhao Mao, Guang Liu, Xiaojie Wang, Weiguo Gao, Xuan Li
Abstract
Emotion dynamics formulates principles explaining the emotional fluctuation during conversations. Recent studies explore the emotion dynamics from the self and inter-personal dependencies, however, ignoring the temporal and spatial dependencies in the situation of multi-modal conversations. To address the issue, we extend the concept of emotion dynamics to multi-modal settings and propose a Dialogue Transformer for simultaneously modeling the intra-modal and inter-modal emotion dynamics. Specifically, the intra-modal emotion dynamics is to not only capture the temporal dependency but also satisfy the context preference in every single modality. The intermodal emotional dynamics aims at handling multi-grained spatial dependency across all modalities. Our models outperform the stateof-the-art with a margin of 4%-16% for most of the metrics on three benchmark datasets.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- A Training-Free Debiasing Framework with Counterfactual Reasoning for Conversational Emotion Detection cites Real-Time Emotion Recognition via Attention Gated Hierarchical Memory Network
- A Training-Free Debiasing Framework with Counterfactual Reasoning for Conversational Emotion Detection cites COSMIC: COmmonSense knowledge for eMotion Identification in Conversations
- ECERC: Evidence-Cause Attention Network for Multi-Modal Emotion Recognition in Conversation cites UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition
- ECERC: Evidence-Cause Attention Network for Multi-Modal Emotion Recognition in Conversation cites COSMIC: COmmonSense knowledge for eMotion Identification in Conversations
- UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition cites COSMIC: COmmonSense knowledge for eMotion Identification in Conversations
- Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition cites UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition
- Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition cites COSMIC: COmmonSense knowledge for eMotion Identification in Conversations
Cited by top-tier papers4
- UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion RecognitionGuimin Hu, Ting-En Lin, Yi Zhao, Guangming Lu et al.EMNLP 2022 · 206 citations
- Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion RecognitionBobo Li, Hao Fei, Lizi Liao, Yu Zhao et al.ACM MM 2023 · 76 citations
- A Training-Free Debiasing Framework with Counterfactual Reasoning for Conversational Emotion DetectionGeng Tu, Ran Jing, Bin Liang, Min Yang et al.EMNLP 2023 · 9 citations
- ECERC: Evidence-Cause Attention Network for Multi-Modal Emotion Recognition in ConversationTao Zhang, Zhenhua TanACL 2025 · 4 citations
Builds on2
- COSMIC: COmmonSense knowledge for eMotion Identification in ConversationsDeepanway Ghosal, Navonil Majumder, Alexander F. Gelbukh, Rada Mihalcea et al.EMNLP Findings 2020 · 390 citations
- Real-Time Emotion Recognition via Attention Gated Hierarchical Memory NetworkWenxiang Jiao, Michael R. Lyu, Irwin KingAAAI 2020 · 149 citations
Related papers
- EmoCaps: Emotion Capsule based Model for Conversational Emotion RecognitionZaijing Li, Fengxiao Tang, Ming Zhao, Yusen ZhuACL Findings 2022 · 117 citations
- Transformer-based Label Set Generation for Multi-modal Multi-label Emotion DetectionXincheng Ju, Dong Zhang, Junhui Li, Guodong ZhouACM MM 2020 · 64 citations
- MIDE: Multimodal Dialogue Emotion Recognition via Mutual Information Enhancement and Dynamic Modality SelectionZhibo Zhang, Jianjun Li, Zhiyuan MaWWW 2026
- Modeling both Intra- and Inter-modal Influence for Real-Time Emotion Detection in ConversationsDong Zhang, Weisheng Zhang, Shoushan Li, Qiaoming Zhu et al.ACM MM 2020 · 34 citations
- Dynamic Interactive Bimodal Hypergraph Networks for Emotion Recognition in ConversationsXuping Chen, Wuzhen ShiAAAI 2025 · 3 citations
