UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis
Junzhi Ning, Wei Li, Cheng Tang, Jiashi Lin, Chenglong Ma, Chaoyang Zhang, Jiyao Liu, Ying Chen, Shujian Gao, Yuandong Pu, Huihui Xu, Chenhui Gou
Abstract
Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central to medical AI. Most existing systems, however, address these abilities in isolated models, losing the shared knowledge that a unified architecture could exploit. To bridge this gap, we present UniMedVL, the first unified medical model that seamlessly integrates multimodal understanding and generation capabilities within a single model without switching weights. We achieve this via a tailored progressive training pipeline where understanding and generation mutually reinforce each other. To effectively train UniMedVL, we curate UniMedVL-5M, the first large-scale medical dataset comprising over 5.6M instances across 8 medical imaging modalities, tailored for multimodal input-output tasks in unified medical understanding and generation. Experimental results demonstrate that UniMedVL achieves competitive performance on five medical understanding benchmarks. Crucially, UniMedVL natively supports diverse interleaved generation tasks, e.g., virtual staining, super-resolution, cross-modal synthesis, essential for complex medical workflows. Our code and dataset are publicly available.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 9d366d40-898f-4419-bcb8-52bef99b4610Cited by top-tier papers4
- Motus: A Unified Latent Action World ModelHongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang et al.CVPR 2026 · 271 citations
- Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype ControlMinghao Han, Yichen Liu, Yizhou Liu, Zizhi Chen et al.CVPR 2026 · 5 citations
- Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and GenerationZhiwang Zhou, Yuandong Pu, Xuming He, Yidi Liu et al.ICLR 2026
- SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task AlignmentWeiren Zhao, DONG Yi, Cheng ChenICML 2026
Builds on16
- NExT-GPT: Any-to-Any Multimodal LLMShengqiong Wu, Hao Fei, Leigang Qu, Wei Ji et al.ICML 2024 · 786 citations
- UniTok: a Unified Tokenizer for Visual Generation and UnderstandingChuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang et al.NeurIPS 2025 · 164 citations
- Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and ActionJiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang et al.CVPR 2024 · 53 citations
- Towards Injecting Medical Visual Knowledge into Multimodal LLMs at ScaleJunying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao et al.EMNLP 2024 · 43 citations
- Label Decoupling and Reconstruction: A Two-Stage Training Framework for Long-tailed Multi-label Medical Image RecognitionJie Huang, Zhao-Min Chen, Xiaoqin Zhang, Yisu Ge et al.ACM MM 2024 · 11 citations
Related papers
- MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for MedicineYunfei Xie, Ce Zhou, Lang Gao, Juncheng Wu et al.ICLR 2025
- UniM: A Unified Any-to-Any Interleaved Multimodal BenchmarkYanlin Li, Minghui Guo, Kaiwen Zhang, Shize Zhang et al.CVPR 2026 · 10 citations
- Beyond Single View: A Comprehensive Benchmark for Medical Multimodal Large Language Models on Multi-Image UnderstandingDexuan Xu, Jiayin Yuan, Jianing Wang, Yanyuan Chen et al.ACL 2026
- MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual InvariantChenlu Zhan, Yu Lin, Gaoang Wang, Hongwei Wang et al.CVPR 2024 · 20 citations
- MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level PrecisionZhonghao Yan, Muxi Diao, Yuxuan Yang, Ruoyan Jing et al.AAAI 2026 · 4 citations
