API: Adaptive Prototype Imputation for Incomplete Multimodal Sentiment Analysis
Xiaotao Wang, Yiyang Fang, Wenke Huang, Bin Yang, Guancheng Wan, Mang Ye
摘要
Multimodal sentiment analysis aims to infer human emotions by integrating signals from diverse modalities. However, missing modalities are common in real-world applications due to sensor failure, data corruption, or privacy concerns. Existing approaches typically follow two main paradigms: recovery-based and non-recovery-based methods. This dichotomy results in two critical limitations: I) computational inefficiency and semantic inconsistency (recovery-based methods rely on heavy generators that incur prohibitive inference latency and risk semantic drift due to lack of class-level priors); II) lack of instance specificity (non-recovery-based methods rely on static global mappings that fail to capture sample-specific affective cues). To address these gaps, we propose Adaptive Prototype Imputation (API). To mitigate I), we introduce Semantic-anchored Class-Temporal Prototype Estimation (SCOPE) to construct non-trainable prototypes as stable semantic anchors, promoting semantic reliability. To resolve II), we design Directional Instance-Adaptive Affine Modulation (DIAM) to dynamically modulate these anchors via direction-specific affine transformations, capturing instance-unique affective characteristics without generative overhead. Experimental results on CMU-MOSI and CMU-MOSEI demonstrate that API outperforms state-of-the-art baselines, establishing a robust and lightweight prototype-centric paradigm for multimodal sentiment analysis. The code is publicly available at https://github.com/KX-yolo/API.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper28
- Video Diffusion ModelsJonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan 等NeurIPS 2022 · 被引用 2,948 次
- MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment AnalysisDevamanyu Hazarika, Roger Zimmermann, Soujanya PoriaACM MM 2020 · 被引用 1,037 次
- On the Efficacy of Knowledge DistillationJang Hyun Cho, Bharath HariharanICCV 2019 · 被引用 741 次
- Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment AnalysisWenmeng Yu, Hua Xu, Ziqi Yuan, Jiele WuAAAI 2021 · 被引用 737 次
- DiffusionDet: Diffusion Model for Object DetectionShoufa Chen, Peize Sun, Yibing Song, Ping LuoICCV 2023 · 被引用 715 次
相关 Paper
- Recovering Coherent Affective Patterns: Addressing Modality Missing in Multimodal Sentiment AnalysisHuiting Huang, Tieliang Gong, Kai He, Wen Wen 等AAAI 2026
- Hyper-Modality Enhancement for Multimodal Sentiment Analysis with Missing ModalitiesYan Zhuang, Minhao Liu, Wei Bai, Yanru Zhang 等NeurIPS 2025 · 被引用 10 次
- Sample-specific Modality Diagnosis and Cross-modal Enhancement for Incomplete Multimodal RepresentationsJunsong Chen, Jiyuan Liu, Suyuan Liu, Wei Zhang 等AAAI 2026
- Mitigating Inconsistencies in Multimodal Sentiment Analysis under Uncertain Missing ModalitiesJiandian Zeng, Jiantao Zhou, Tianyi LiuEMNLP 2022 · 被引用 29 次
- TMDC: A Two-Stage Modality Denoising and Complementation Framework for Multimodal Sentiment Analysis with Missing and Noisy ModalitiesYan Zhuang, Minhao Liu, Yanru Zhang, Jiawen Deng 等AAAI 2026 · 被引用 2 次
