UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis
Junzhi Ning, Wei Li, Cheng Tang, Jiashi Lin, Chenglong Ma, Chaoyang Zhang, Jiyao Liu, Ying Chen, Shujian Gao, Yuandong Pu, Huihui Xu, Chenhui Gou
摘要
Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central to medical AI. Most existing systems, however, address these abilities in isolated models, losing the shared knowledge that a unified architecture could exploit. To bridge this gap, we present UniMedVL, the first unified medical model that seamlessly integrates multimodal understanding and generation capabilities within a single model without switching weights. We achieve this via a tailored progressive training pipeline where understanding and generation mutually reinforce each other. To effectively train UniMedVL, we curate UniMedVL-5M, the first large-scale medical dataset comprising over 5.6M instances across 8 medical imaging modalities, tailored for multimodal input-output tasks in unified medical understanding and generation. Experimental results demonstrate that UniMedVL achieves competitive performance on five medical understanding benchmarks. Crucially, UniMedVL natively supports diverse interleaved generation tasks, e.g., virtual staining, super-resolution, cross-modal synthesis, essential for complex medical workflows. Our code and dataset are publicly available.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper4
- Motus: A Unified Latent Action World ModelHongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang 等CVPR 2026 · 被引用 271 次
- Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype ControlMinghao Han, Yichen Liu, Yizhou Liu, Zizhi Chen 等CVPR 2026 · 被引用 5 次
- Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and GenerationZhiwang Zhou, Yuandong Pu, Xuming He, Yidi Liu 等ICLR 2026
- SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task AlignmentWeiren Zhao, DONG Yi, Cheng ChenICML 2026
它引用的顶会 Paper16
- NExT-GPT: Any-to-Any Multimodal LLMShengqiong Wu, Hao Fei, Leigang Qu, Wei Ji 等ICML 2024 · 被引用 786 次
- UniTok: a Unified Tokenizer for Visual Generation and UnderstandingChuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang 等NeurIPS 2025 · 被引用 164 次
- Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and ActionJiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang 等CVPR 2024 · 被引用 53 次
- Towards Injecting Medical Visual Knowledge into Multimodal LLMs at ScaleJunying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao 等EMNLP 2024 · 被引用 43 次
- Label Decoupling and Reconstruction: A Two-Stage Training Framework for Long-tailed Multi-label Medical Image RecognitionJie Huang, Zhao-Min Chen, Xiaoqin Zhang, Yisu Ge 等ACM MM 2024 · 被引用 11 次
相关 Paper
- MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for MedicineYunfei Xie, Ce Zhou, Lang Gao, Juncheng Wu 等ICLR 2025
- UniM: A Unified Any-to-Any Interleaved Multimodal BenchmarkYanlin Li, Minghui Guo, Kaiwen Zhang, Shize Zhang 等CVPR 2026 · 被引用 10 次
- Beyond Single View: A Comprehensive Benchmark for Medical Multimodal Large Language Models on Multi-Image UnderstandingDexuan Xu, Jiayin Yuan, Jianing Wang, Yanyuan Chen 等ACL 2026
- MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual InvariantChenlu Zhan, Yu Lin, Gaoang Wang, Hongwei Wang 等CVPR 2024 · 被引用 20 次
- MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level PrecisionZhonghao Yan, Muxi Diao, Yuxuan Yang, Ruoyan Jing 等AAAI 2026 · 被引用 4 次
