Lune

CVPR2026Top-tier venue

Beyond Missing Modalities: Hypergraph Conditioned Diffusion for Uncertainty-Aware Multimodal Emotion Recognition

Xihang Qiu, Yuhao Fang, Qing Zhou, Bin Zhai, Jialong Hong, Wanpeng Zhang, Yao Lu, Ye Zhang, Chun Li

Abstract

Multimodal Emotion Recognition in Conversations (MERC) aims to understand emotions expressed in each utterance by effectively integrating audio, text, and visual modalities. However, in real-world scenarios, unavoidable missing modalities often degrade multimodal interpretation performance. To address this, we propose Hypergraph Diffusion and Evidence Fusion based Emotion Recognition (HyperEF), a novel framework designed to mitigate challenges arising from incomplete modalities in MERC. Specifically, to mitigate performance degradation caused by modality absence, we propose Masked Hypergraph Attention (MHGAT) conditioned diffusion model to recover latent features of missing modalities in the latent space. To ensure semantic consistency between recovered and available modalities within the same utterance, we introduce MHGAT that captures high-order semantic information from available modalities to guide the diffusion model's denoising process. Furthermore, to disentangle and model the complex uncertainties inherent in MERC, we propose Dual Channel Evidence Fusion (DCEF), which estimates uncertainty at both feature source level and discriminative level, thereby achieving adaptive evidence fusion. Extensive comparative experiments and interpretability demonstrate the superior performance of our model in emotion recognition, as well as the contribution of each module within the model.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning cites Incomplete Multimodality-Diffused Emotion Recognition
  • Incomplete Multimodality-Diffused Emotion Recognition cites Distribution-Consistent Modal Recovering for Incomplete Multimodal Learning
  • Incomplete Multimodality-Diffused Emotion Recognition cites Score-Based Generative Modeling through Stochastic Differential Equations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
  2. Score-Based Generative Modeling through Stochastic Differential EquationsYang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar et al.ICLR 2021 · 1,270 citations
  3. Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction TuningZebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Kai Wang et al.NeurIPS 2024 · 293 citations
  4. Incomplete Multimodality-Diffused Emotion RecognitionYuanzhi Wang, Yong Li, Zhen CuiNeurIPS 2023 · 155 citations
  5. Distribution-Consistent Modal Recovering for Incomplete Multimodal LearningYuanzhi Wang, Zhen Cui, Yong LiICCV 2023 · 101 citations

Related papers

  1. Federated Dialogue-Semantic Diffusion for Emotion Recognition under Incomplete ModalitiesXihang Qiu, Jiarong Cheng, Yuhao Fang, Wanpeng Zhang et al.NeurIPS 2025 · 1 citation
  2. CaM-HG: Causal-Enhanced MoE and Hypergraphs Network for Incomplete Multimodal Emotion Recognition in ConversationsMingjian Yang, Yong Wang, Peng Liu, Wen YinACL Findings 2026
  3. Hyper-Modality Enhancement for Multimodal Sentiment Analysis with Missing ModalitiesYan Zhuang, Minhao Liu, Wei Bai, Yanru Zhang et al.NeurIPS 2025 · 10 citations
  4. Multimodal Fusion via Hypergraph Autoencoder and Contrastive Learning for Emotion Recognition in ConversationZijian Yi, Ziming Zhao, Zhishu Shen, Tiehua ZhangACM MM 2024 · 32 citations
  5. ECERC: Evidence-Cause Attention Network for Multi-Modal Emotion Recognition in ConversationTao Zhang, Zhenhua TanACL 2025 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines