CVPR Findings2026
3DFA: Aligning the Features Between Point Cloud and Query Image for Scene-Specific Visual Localization
Sizhe Song, Yankuan Chi, Shuhan Zhong, S.-H. Gary Chan
Abstract
We consider scene-specific visual localization, i.e., estimating the camera poses of a query image taken in a specific site given the training data composed of support images in the same site and their corresponding reconstructed 3D point cloud. A critical performance factor in the localization pipeline is aligning the features of the 2D query image with the training data for the subsequent keypoint matching with the point cloud. Prior art has been primarily focusing on aligning the query images with the support images without making the most of the rich geometry and appearance information of the point cloud. To leverage the point cloud information so as to achieve higher localization accuracy, we propose 3DFA, a novel approach which directly aligns the 2D query image with the features of the 3D model. 3DFA employs a simple yet effective hierarchical patching operation designed for direct 2D-3D alignment, and a patch-matching loss without the need for any additional annotation. Our extensive experiments on representative datasets demonstrate that 3DFA outperforms the SOTA methods substantially, cutting the translation and rotation errors by 18.3% and 8.4%, respectively, and achieving robustness against different levels of scene noise.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on16
- ScanNet++: A High-Fidelity Dataset of 3D Indoor ScenesChandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, Angela DaiICCV 2023 · 659 citations
- Point-NeRF: Point-based Neural Radiance FieldsQiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi et al.CVPR 2022 · 510 citations
- DUSt3R: Geometric 3D Vision Made EasyShuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii et al.CVPR 2024 · 302 citations
- EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose EstimationHansheng Chen, Pichao Wang, Fan Wang, Wei Tian et al.CVPR 2022 · 175 citations
- Learning Multi-Scene Absolute Pose Regression with TransformersYoli Shavit, Ron Ferens, Yosi KellerICCV 2021 · 163 citations
Related papers
- EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual LocalizationMinjung Kim, Junseo Koo, Gunhee KimICCV 2023 · 22 citations
- Learning to Produce Semi-Dense Correspondences for Visual LocalizationKhang Truong Giang, Soohwan Song, Sungho JoCVPR 2024
- How Privacy-Preserving Are Line Clouds? Recovering Scene Details From 3D LinesKunal Chelani, Fredrik Kahl, Torsten SattlerCVPR 2021
- SAG-GNN: Semantic-Aware Guided GNN for Descriptor-Free 2D-3D MatchingShihua Zhang, Tianhao Xu, Zizhuo Li, Qing Ma et al.CVPR 2026
- From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian SplattingZhiwei Huang, Hailin Yu, Yichun Shentu, Jin Yuan et al.CVPR 2025
