Cross-Modal Guided Visual Synthesis for Data-Efficient Multimodal Depression Recognition
Shanliang Yang, Xiaoxiao Wang
摘要
The performance of multimodal learning systems, particularly in high-stakes domains like automated depression recognition, is fundamentally constrained by the challenge of learning robust visual representations from limited and complex clinical data. To overcome this, we introduce Cross-Modal Guided Visual Synthesis (CMG-VS), a novel training framework that internally enhances the learning process by synthesizing new, task-relevant visual features. At its core, CMG-VS leverages the rich context from audio and text modalities to guide a conditional generative model. This model learns the intricate mapping from speech and language to visual expression, generating a diverse manifold of plausible visual behaviors to enrich the training distribution. Crucially, this synthesis is not a separate pre-processing step. Through a task-guided joint optimization scheme, the generative process is dynamically steered by the downstream multimodal recognizer's performance. This closed-loop feedback mechanism ensures the synthesized visual features are optimized to be maximally discriminative for the recognition task, rather than merely realistic. Comprehensive experiments on the widely-used DAIC-WOZ and E-DAIC benchmark datasets demonstrate that CMG-VS significantly outperforms existing state-of-the-art methods across all standard regression and classification metrics. Ablation studies further validate that our task-guided synthesis is the key driver of this performance gain, proving its effectiveness as a new paradigm for robust multimodal representation learning.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper7
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li 等NeurIPS 2022 · 被引用 8,965 次
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable FeaturesSangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh 等ICCV 2019 · 被引用 5,843 次
- CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression EstimationHao Sun, Hongyi Wang, Jiaqing Liu, Yen-Wei Chen 等ACM MM 2022 · 被引用 158 次
- FacialPulse: An Efficient RNN-based Depression Detection via Temporal Facial LandmarksRuiqi Wang, Jinyang Huang, Jie Zhang, Xin Liu 等ACM MM 2024 · 被引用 22 次
- Diffusemix: Label-Preserving Data Augmentation with Diffusion ModelsKhawar Islam, Muhammad Zaigham Zaheer, Arif Mahmood, Karthik NandakumarCVPR 2024
相关 Paper
- Boosting Masked ECG-Text Auto-Encoders as Discriminative LearnersHung Manh Pham, Aaqib Saeed, Dong MaICML 2025
- Unveiling the Landscape of Clinical Depression Assessment: From Behavioral Signatures to Psychiatric ReasoningZhuang Chen, Guanqun Bi, Wen Zhang, Jiawei Hu 等AAAI 2026 · 被引用 2 次
- Learning to See through Sound: From VggCaps to Multi2Cap for Richer Automated Audio CaptioningSangyeon Cho, Mingi Kim, Jinkwon Hwang, Jaehoon Go 等EMNLP 2025
- When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression DetectionXiangyu Zhang, Hexin Liu, Kaishuai Xu, Qiquan Zhang 等EMNLP 2024 · 被引用 13 次
- Auto-ACD: A Large-scale Dataset for Audio-Language Representation LearningLuoyi Sun, Xuenan Xu, Mengyue Wu, Weidi XieACM MM 2024 · 被引用 23 次
