Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment
Chen Liu, Wenfang Yao, Kejing Yin, William K. Cheung, Jing Qin
Abstract
Longitudinal multimodal data, including electronic health records (EHR) and sequential chest X-rays (CXRs), is critical for modeling disease progression, yet remains underutilized due to two key challenges: (1) redundancy in consecutive CXR sequences, where static anatomical regions dominate over clinically-meaningful dynamics, and (2) temporal misalignment between sparse, irregular imaging and continuous EHR data. We introduce , a novel framework that addresses these challenges through region-aware disentanglement and multi-timescale alignment. First, we disentangle static (anatomy) and dynamic (pathology progression) features in sequential CXRs, prioritizing disease-relevant changes. Second, we hierarchically align these static and dynamic CXR features with asynchronous EHR data via local (pairwise interval-level) and global (full-sequence) synchronization to model coherent progression pathways. Extensive experiments on the MIMIC dataset demonstrate that could effectively extract temporal clinical dynamics and achieve state-of-the-art performance on both disease progression identification and general ICU prediction tasks.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on11
- Quantifying & Modeling Multimodal Interactions: An Information Decomposition FrameworkPaul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling et al.NeurIPS 2023 · 120 citations
- DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal InconsistencyWenfang Yao, Kejing Yin, William K. Cheung, Jia Liu et al.AAAI 2024 · 80 citations
- Improving Medical Predictions by Irregular Multimodal Electronic Health Records ModelingXinlu Zhang, Shiyang Li, Zhiyu Chen, Xifeng Yan et al.ICML 2023 · 54 citations
- Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage AnalysisMengwei Ren, Neel Dey, Martin Styner, Kelly N. Botteron et al.NeurIPS 2022 · 26 citations
- Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray GenerationWenfang Yao, Chen Liu, Kejing Yin, William Kwok-Wai Cheung et al.NeurIPS 2024 · 11 citations
Related papers
- LUMIN: A Longitudinal Multi-modal Knowledge Decomposition Network for Predicting Breast Cancer RecurrenceChunyao Lu, Tianyu Zhang, Xinglong Liang, Yuan Gao et al.AAAI 2026
- The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate ItAaron Nicolson, Shengyao Zhuang, Jason Dowling, Bevan KoopmanACL 2025 · 6 citations
- FlexCare: Leveraging Cross-Task Synergy for Flexible Multimodal Healthcare PredictionMuhao Xu, Zhenfeng Zhu, Youru Li, Shuai Zheng et al.KDD 2024 · 5 citations
- Temporal Inversion for Learning Interval Change in Chest X-RaysHanbin Ko, Kyeongmin Jeon, Doowoong Choi, Chang Min ParkCVPR 2026 · 3 citations
- BiOTPrompt: Bidirectional Optimal Transport Guided Prompting for Disease Evolution-aware Radiology Report GenerationTengfei Liu, Yijian Fan, Boyue Wang, Yongli Hu et al.CVPR 2026
