Scaling Spatial Intelligence with Multimodal Foundation Models
Zhongang Cai, Wang Ruisi, Chenyang Gu, Fanyi Pu, Junxiang Xu, YUBO WANG, Wanqi Yin, Zhitao Yang, Chen Wei, Tongxi Zhou, Qingping SUN, Hui En Pang
摘要
Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family*, built upon established multimodal foundations including visual understanding models (i.e., Qwen3-VL and InternVL3) and unified understanding and generation models (i.e., Bagel). We take a principled approach to constructing high-performing and robust spatial intelligence by systematically curating SenseNova-SI-8M: eight million diverse data samples under a rigorous taxonomy of spatial capabilities. SenseNova-SI demonstrates unprecedented performance across a broad range of spatial intelligence benchmarks: 68.8% on VSI-Bench, 43.3% on MMSI, 85.7% on MindCube, 54.7% on ViewSpatial, 47.7% on SITE, 63.9% on BLINK, 55.5% on 3DSR, and 72.0% on EmbSpatial, while maintaining strong general multimodal understanding (e.g., 84.9% on MMBench-En). More importantly, we analyze the impact of data scaling, discuss early signs of emergent generalization capabilities enabled by diverse data training, analyze the risk of overfitting and language shortcuts, present a preliminary study on spatial chain-of-thought reasoning, and validate the potential downstream application. All newly trained multimodal foundation models are publicly released.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper9
- SpatialScore: Towards Comprehensive Evaluation for Spatial IntelligenceHaoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang 等CVPR 2026 · 被引用 12 次
- Thinking with Geometry: Active Geometry Integration for Spatial ReasoningHaoyuan Li, Qihang Cao, Tao Tang, Kun Xiang 等ICML 2026 · 被引用 12 次
- On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMsRosie Zhao, Anshul Shah, Xiaoyu Zhu, Xinke Deng 等ICML 2026 · 被引用 10 次
- Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial IntelligenceYuanyuan Gao, Hao Li, Yifei Liu, Xinhao Ji 等ICML 2026 · 被引用 5 次
- HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language ModelsHuizhi Liang, Yichao Shen, Yu Deng, Sicheng Xu 等CVPR 2026 · 被引用 2 次
它引用的顶会 Paper32
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- Are We on the Right Way for Evaluating Large Vision-Language Models?Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang 等NeurIPS 2024 · 被引用 1,029 次
- ScanNet++: A High-Fidelity Dataset of 3D Indoor ScenesChandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, Angela DaiICCV 2023 · 被引用 659 次
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language ModelsAn-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo 等NeurIPS 2024 · 被引用 412 次
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial IntelligenceDiankun Wu, Fangfu Liu, Yi-Hsin Hung, Yueqi DuanNeurIPS 2025 · 被引用 245 次
相关 Paper
- MMSI-Bench: A Benchmark for Multi-Image Spatial IntelligenceSihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang 等ICLR 2026 · 被引用 195 次
- SITE: Towards Spatial Intelligence Thorough EvaluationWenqi Wang, Reuben Tan, Pengyue Zhu, Jianwei Yang 等ICCV 2025 · 被引用 2 次
- InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language ModelsNianchen Deng, Lixin Gu, Shenglong Ye, Yinan He 等ICLR 2026 · 被引用 32 次
- Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot StudyGuanlin Wu, Boyan Su, Yang Zhao, Pu Wang 等NeurIPS 2025 · 被引用 2 次
- SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language ModelsHongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan 等ICLR 2026 · 被引用 75 次
