AffIn-Space: Learning Affine-Invariant Representations for 3D Spatial Understanding with MLLMs
Zhenyu Lu, Liupeng Li, Jinpeng Wang, Haoqian Kang, Manyuan Zhang, Yan Feng, Ke Chen, Yaowei Wang
摘要
While MLLMs show promising capacity on general visual understanding, they suffer from geometric fragility : standard visual representations often degrade rapidly under changes in viewpoint and viewing distance. Our analysis identifies that existing paradigms, whether relying on input-level fusion or latent reconstruction, remain entangled with the view-dependent pixel grid, failing to decouple intrinsic 3D structure from extrinsic camera pose. To address this, we introduce AffIn-Space , a framework that enforces strict affine invariance to enable robust spatial understanding. Unlike implicit learning approaches, AffIn-Space introduces a two-stage explicit decoupling mechanism. First, it employs explicit geometric resampling by utilizing decomposed affine quantities (derived from pose features) to spatially align 3D features to a canonical state before fusion. Second, within the MLLM, we implement affine-invariant constraints via an orthogonal projection mechanism, which mathematically strips away pose-dependent noise from the hidden states while retaining recoverable geometric semantics through conditional reconstruction. Extensive experiments on VSI-Bench, ScanQA, SQA3D, Scan2Cap, and EmbodiedScan demonstrate that AffIn-Space achieves state-of-the-art performance. Code and detailed instructions will be publicly released. Crucially, our approach exhibits superior stability against affine perturbations, validating the effectiveness of explicitly modeling geometric invariance for complex spatial tasks. Code will be made available. Extensive experiments show that AffIn-Space achieves state-of-the-art performance on spatial reasoning tasks (VSI-Bench, SQA3D and Scan2Cap), and on spatial grounding tasks (ScanRefer and EmbodiedScan), demonstrating the effectiveness of affine invariant representations for complex spatial understanding.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper22
- 3D-LLM: Injecting the 3D World into Large Language ModelsYining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng 等NeurIPS 2023 · 被引用 662 次
- π3: Permutation-Equivariant Visual Geometry LearningYifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang 等ICLR 2026 · 被引用 318 次
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial IntelligenceDiankun Wu, Fangfu Liu, Yi-Hsin Hung, Yueqi DuanNeurIPS 2025 · 被引用 245 次
- Chat-Scene: Bridging 3D Scene and Large Language Models with Object IdentifiersHaifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang 等NeurIPS 2024 · 被引用 230 次
- ScanQA: 3D Question Answering for Spatial Scene UnderstandingDaichi Azuma, Taiki Miyanishi, Shuhei Kurita, Motoaki KawanabeCVPR 2022 · 被引用 135 次
相关 Paper
- SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language ModelsRuosen Zhao, Zhikang Zhang, Jialei Xu, Jiahao Chang 等CVPR 2026 · 被引用 21 次
- G^2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial ReasoningWenbo hu, JINGLI LIN, Yilin Long, Yunlong Ran 等CVPR 2026
- S^2-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural GuidanceBeining Xu, Siting Zhu, Zhao Jin, Junxian Li 等CVPR 2026
- On the Generalization Capacities of MLLMs for Spatial IntelligenceGongjie Zhang, Wenhao Li, Quanhao Qian, Jiuniu Wang 等ICLR 2026 · 被引用 10 次
- Thinking with Geometry: Active Geometry Integration for Spatial ReasoningHaoyuan Li, Qihang Cao, Tao Tang, Kun Xiang 等ICML 2026 · 被引用 12 次
