Virtual Full-stack Scanning of Brain MRI via Imputing Any Quantised Code
Yicheng Wu, Tao Song, Zhonghua Wu, Jin Ye, Zongyuan Ge, Wenjia Bai, Zhaolin Chen, Jianfei Cai
Abstract
Magnetic resonance imaging (MRI) is a powerful and versatile imaging technique, offering a wide spectrum of information about the anatomy by employing different acquisition modalities. However, in the clinical workflow, it is impractical to collect all relevant modalities due to the scan time and cost constraints. Virtual full-stack scanning aims to impute missing MRI modalities from available but incomplete acquisitions, offering a cost-efficient solution to enhance data completeness and clinical usability. Existing imputation methods often depend on global conditioning or modality-specific designs, which limit their generalisability across patient cohorts and imaging protocols. To address these limitations, we propose CodeBrain, a unified framework that reformulates various ``any-to-any''imputation tasks as a region-level full-stack code prediction problem. CodeBrain adopts a two-stage pipeline: (1) it learns the compact representation of a complete MRI modality set by encoding it into scalar-quantised codes at the region level, enabling high-fidelity image reconstruction after decoding these codes along with modality-agnostic common features; (2) it trains a projection encoder to predict the full-stack code map from incomplete modalities via a grading-based design for diverse imputation scenarios. Extensive experiments on two public brain MRI datasets, i.e., IXI and BraTS 2023, demonstrate that CodeBrain consistently outperforms state-of-the-art methods, establishing a new benchmark for unified brain MRI imputation and enabling virtual full-stack scanning. Our code will be released at https://github.com/ycwu1997/CodeBrain.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- High-Resolution Image Synthesis with Latent Diffusion Models cites Vector-quantized Image Modeling with Improved VQGAN
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction cites High-Resolution Image Synthesis with Latent Diffusion Models
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction cites Vector-quantized Image Modeling with Improved VQGAN
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction cites Finite Scalar Quantization: VQ-VAE Made Simple
- Finite Scalar Quantization: VQ-VAE Made Simple cites Vector-quantized Image Modeling with Improved VQGAN
Builds on18
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale PredictionKeyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng et al.NeurIPS 2024 · 1,199 citations
- Vector-quantized Image Modeling with Improved VQGANJiahui Yu, Xin Li, Jing Yu Koh, Han Zhang et al.ICLR 2022 · 753 citations
- Vector Quantized Diffusion Model for Text-to-Image SynthesisShuyang Gu, Dong Chen, Jianmin Bao, Fang Wen et al.CVPR 2022 · 607 citations
- Finite Scalar Quantization: VQ-VAE Made SimpleFabian Mentzer, David Minnen, Eirikur Agustsson, Michael TschannenICLR 2024 · 442 citations
Related papers
- Modality-Aware and Anatomical Vector-Quantized Autoencoding for Multimodal Brain MRIMingjie Li, Edward Kim, Yue Zhao, Ehsan Adeli, Kilian M. PohlCVPR Findings 2026
- Virtual Nodes Guided Dynamic Graph Neural Network for Brain Tumor Segmentation with Missing ModalitiesSha Tao, Jiao Pan, Yu Guo, Chao YaoCVPR 2026
- Uni-Encoder Meets Multi-Encoders: Representation Before Fusion for Brain Tumor Segmentation with Missing ModalitiesPeibo Song, Xiaotian Xue, Jinshuo Zhang, Zihao Wang et al.CVPR 2026
- Auto-GAN: Self-Supervised Collaborative Learning for Medical Image SynthesisBing Cao, Han Zhang, Nannan Wang, Xinbo Gao et al.AAAI 2020 · 94 citations
- KMD: Koopman Multi-modality Decomposition for Generalized Brain Tumor Segmentation under Incomplete ModalitiesTianyi Liu, Haochuan Jiang, Kaizhu HuangCVPR 2025
