Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and Generation
Zhiwang Zhou, Yuandong Pu, Xuming He, Yidi Liu, Yixin Chen, Junchao Gong, Xiang Zhuang, Wanghan Xu, Qinglong Cao, Shixiang Tang, Yihao Liu, Wenlong Zhang, Lei Bai
摘要
Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper16
- ClimaX: A foundation model for weather and climateTung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta 等ICML 2023 · 被引用 426 次
- Earthformer: Exploring Space-Time Transformers for Earth System ForecastingZhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu 等NeurIPS 2022 · 被引用 410 次
- SEVIR : A Storm Event Imagery Dataset for Deep Learning Applications in Radar and Satellite MeteorologyMark S. Veillette, Siddharth Samsi, Christopher J. MattioliNeurIPS 2020 · 被引用 179 次
- PreDiff: Precipitation Nowcasting with Latent Diffusion ModelsZhihan Gao, Xingjian Shi, Boran Han, Hao Wang 等NeurIPS 2023 · 被引用 171 次
- LMFusion: Adapting Pretrained Language Models for Multimodal GenerationWeijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang 等NeurIPS 2025 · 被引用 134 次
相关 Paper
- WeatherGFM: Learning a Weather Generalist Foundation Model via In-context LearningXiangyu Zhao, Zhiwang Zhou, Wenlong Zhang, Yihao Liu 等ICLR 2025
- RadarQA: Multi-modal Quality Analysis of Weather Radar ForecastsXuming He, Zhiyuan You, Junchao Gong, Couhua Liu 等NeurIPS 2025 · 被引用 14 次
- OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time ScalesTung Nguyen, Tuan Pham, Troy Arcomano, Rao Kotamarthi 等NeurIPS 2025 · 被引用 12 次
- VILA-U: a Unified Foundation Model Integrating Visual Understanding and GenerationYecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang 等ICLR 2025
- Show-o: One Single Transformer to Unify Multimodal Understanding and GenerationJinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang 等ICLR 2025
