API: Adaptive Prototype Imputation for Incomplete Multimodal Sentiment Analysis
Xiaotao Wang, Yiyang Fang, Wenke Huang, Bin Yang, Guancheng Wan, Mang Ye
Abstract
Multimodal sentiment analysis aims to infer human emotions by integrating signals from diverse modalities. However, missing modalities are common in real-world applications due to sensor failure, data corruption, or privacy concerns. Existing approaches typically follow two main paradigms: recovery-based and non-recovery-based methods. This dichotomy results in two critical limitations: I) computational inefficiency and semantic inconsistency (recovery-based methods rely on heavy generators that incur prohibitive inference latency and risk semantic drift due to lack of class-level priors); II) lack of instance specificity (non-recovery-based methods rely on static global mappings that fail to capture sample-specific affective cues). To address these gaps, we propose Adaptive Prototype Imputation (API). To mitigate I), we introduce Semantic-anchored Class-Temporal Prototype Estimation (SCOPE) to construct non-trainable prototypes as stable semantic anchors, promoting semantic reliability. To resolve II), we design Directional Instance-Adaptive Affine Modulation (DIAM) to dynamically modulate these anchors via direction-specific affine transformations, capturing instance-unique affective characteristics without generative overhead. Experimental results on CMU-MOSI and CMU-MOSEI demonstrate that API outperforms state-of-the-art baselines, establishing a robust and lightweight prototype-centric paradigm for multimodal sentiment analysis. The code is publicly available at https://github.com/KX-yolo/API.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 46c168a0-d787-484a-97f5-71699fc952c8Builds on28
- Video Diffusion ModelsJonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan et al.NeurIPS 2022 · 2,948 citations
- MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment AnalysisDevamanyu Hazarika, Roger Zimmermann, Soujanya PoriaACM MM 2020 · 1,037 citations
- On the Efficacy of Knowledge DistillationJang Hyun Cho, Bharath HariharanICCV 2019 · 741 citations
- Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment AnalysisWenmeng Yu, Hua Xu, Ziqi Yuan, Jiele WuAAAI 2021 · 737 citations
- DiffusionDet: Diffusion Model for Object DetectionShoufa Chen, Peize Sun, Yibing Song, Ping LuoICCV 2023 · 715 citations
Related papers
- Recovering Coherent Affective Patterns: Addressing Modality Missing in Multimodal Sentiment AnalysisHuiting Huang, Tieliang Gong, Kai He, Wen Wen et al.AAAI 2026
- Hyper-Modality Enhancement for Multimodal Sentiment Analysis with Missing ModalitiesYan Zhuang, Minhao Liu, Wei Bai, Yanru Zhang et al.NeurIPS 2025 · 10 citations
- Sample-specific Modality Diagnosis and Cross-modal Enhancement for Incomplete Multimodal RepresentationsJunsong Chen, Jiyuan Liu, Suyuan Liu, Wei Zhang et al.AAAI 2026
- Mitigating Inconsistencies in Multimodal Sentiment Analysis under Uncertain Missing ModalitiesJiandian Zeng, Jiantao Zhou, Tianyi LiuEMNLP 2022 · 29 citations
- TMDC: A Two-Stage Modality Denoising and Complementation Framework for Multimodal Sentiment Analysis with Missing and Noisy ModalitiesYan Zhuang, Minhao Liu, Yanru Zhang, Jiawen Deng et al.AAAI 2026 · 2 citations
