Cambrian-S: Towards Spatial Supersensing in Video
Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, Rob Fergus, Yann LeCun
摘要
We argue that progress in true multimodal intelligence calls for a shift from reactive, taskdriven systems and brute-force long context towards a broader paradigm of supersensing. We frame spatial supersensing as four stages beyond linguistic-only understanding: semantic perception (naming what is seen), streaming event cognition (maintaining memory across continuous experiences), implicit 3D spatial cognition (inferring the world behind pixels), and predictive world modeling (creating internal models that filter and organize information). Current benchmarks largely test only the early stages, offering narrow coverage of spatial cognition and rarely challenging models in ways that require true world modeling. To drive progress in spatial supersensing, we present V S I -S U P E R, a two-part benchmark: VSR (longhorizon visual spatial recall) and VSC (continual visual spatial counting). These tasks require arbitrarily long video inputs yet are resistant to brute-force context expansion. We then test data scaling limits by curating VSI-590K and training Cambrian-S, achieving +30% absolute improvement on VSI-Bench without sacrificing general capabilities. Yet performance on V S I -S U P E R remains limited, indicating that scale alone is insufficient for spatial supersensing. We propose predictive sensing as a path forward, presenting a proof-of-concept in which a selfsupervised next-latent-frame predictor leverages surprise (prediction error) to drive memory and event segmentation. On V S I -S U P E R, this approach substantially outperforms leading proprietary baselines, showing that spatial supersensing requires models that not only see but also anticipate, select, and organize experience.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper25
- Scaling Spatial Intelligence with Multimodal Foundation ModelsZhongang Cai, Wang Ruisi, Chenyang Gu, Fanyi Pu 等CVPR 2026 · 被引用 81 次
- QUANTIPHY: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language ModelsLi Puyin, Tiange Xiang, Ella Mao, Shirley Wei 等CVPR 2026 · 被引用 23 次
- SpatialStack: Layered Geometry-Language Fusion for 3D VLM Spatial ReasoningJian Zhang, Shijie Zhou, Bangya Liu, Achuta Kadambi 等CVPR 2026 · 被引用 16 次
- Utonia: Toward One Encoder for All Point CloudsYujia Zhang, Xiaoyang Wu, Yunhan Yang, Xianzhe Fan 等ICML 2026 · 被引用 12 次
- SpatialScore: Towards Comprehensive Evaluation for Spatial IntelligenceHaoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang 等CVPR 2026 · 被引用 12 次
它引用的顶会 Paper63
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 被引用 64,255 次
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 被引用 7,873 次
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra 等NeurIPS 2022 · 被引用 5,493 次
相关 Paper
- Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMsPeter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo 等NeurIPS 2024 · 被引用 1,004 次
- Thinking with Geometry: Active Geometry Integration for Spatial ReasoningHaoyuan Li, Qihang Cao, Tao Tang, Kun Xiang 等ICML 2026 · 被引用 12 次
- From Where Things Are to What They Are For: Benchmarking Spatial–Functional Intelligence in Multimodal LLMsLe Zhang, Jihan Yang, Soundarya Krishnan, Jimit Majmudar 等CVPR 2026 · 被引用 2 次
- Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video StreamsYun Wang, Junbin Xiao, Han Lyu, Yifan Wang 等ICML 2026
- Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall SpacesJihan Yang, Shusheng Yang, Anjali W. Gupta, Rilyn Han 等CVPR 2025
