Guiding Local Feature Matching with Surface Curvature

We propose a new method, called curvature similarity extractor (CSE), for improving local feature matching across images. CSE calculates the curvature of the local 3D surface patch for each detected feature point in a viewpointinvariant manner via fitting quadrics to predicted monocular depth maps. This curvature is then leveraged as an additional signal in feature matching with off-the-shelf matchers like SuperGlue and LoFTR. Additionally, CSE enables endto-end joint training by connecting the matcher and depth predictor networks. Our experiments demonstrate on largescale real-world datasets that CSE consistently improves the accuracy of state-of-the-art methods. Fine-tuning the depth prediction network further enhances the accuracy. The proposed approach achieves state-of-the-art results on the ScanNet dataset, showcasing the effectiveness of incorporating 3D geometric information into feature matching. 1 * Part of the work was done during the author's visit to ETH zurich. 1 Code and trained models are available at https://github.com/ AaltoVision/surface-curvature-estimator.