Emergent Extreme-View Geometry in 3D Foundation Models
Yiwen Zhang, Joseph Tung, Ruojin Cai, David Fouhey, Hadar Averbuch-Elor
摘要
3D foundation models (3DFMs) have recently transformed 3D vision, enabling joint prediction of depths, poses, and point maps directly from images. Yet their ability to reason under extreme, non-overlapping views remains largely unexplored. In this work, we study their internal representations and find that 3DFMs exhibit an emergent understanding of extreme-view geometry, despite never being trained for such conditions. To further enhance these capabilities, we introduce a lightweight alignment scheme that refines their internal 3D representation by tuning only a small subset of backbone bias terms, leaving all decoder heads frozen. This targeted adaptation substantially improves relative pose estimation under extreme viewpoints without degrading per-image depth or point quality. Additionally, we contribute MegaUnScene, a new benchmark of Internet scenes unseen by existing 3DFMs, with dedicated test splits for both relative pose estimation and dense 3D reconstruction. All code and data will be released.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper32
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- SegFormer: Simple and Efficient Design for Semantic Segmentation with TransformersEnze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar 等NeurIPS 2021 · 被引用 9,661 次
- LightGlue: Local Feature Matching at Light SpeedPhilipp Lindenberger, Paul-Edouard Sarlin, Marc PollefeysICCV 2023 · 被引用 936 次
- DUSt3R: Geometric 3D Vision Made EasyShuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii 等CVPR 2024 · 被引用 302 次
- Neural RGB-D Surface ReconstructionDejan Azinovic, Ricardo Martin-Brualla, Dan B. Goldman, Matthias Nießner 等CVPR 2022 · 被引用 272 次
相关 Paper
- Relative Pose Estimation through Affine Corrections of Monocular Depth PriorsYifan Yu, Shaohui Liu, Rémi Pautrat, Marc Pollefeys 等CVPR 2025
- Dens3R: A Foundation Model for 3D Geometry PredictionXianze Fang, Jingnan Gao, Zhe Wang, Zhuo Chen 等ICLR 2026 · 被引用 45 次
- SegMASt3R: Geometry Grounded Segment MatchingRohit Jayanti, Swayam Agrawal, Vansh Garg, Siddharth Tourani 等NeurIPS 2025 · 被引用 6 次
- Selfi: Self-improving Reconstruction Engine via 3D Geometric Feature AlignmentYouming Deng, Songyou Peng, Junyi Zhang, Kathryn Heal 等CVPR 2026 · 被引用 4 次
- MuM: Multi-View Masked Image Modeling for 3D VisionDavid Nordström, Johan Edstedt, Fredrik Kahl, Georg BökmanCVPR 2026 · 被引用 6 次
