Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares

uncertainty estimates from images regress uncertainty estimates from pose error images with correspondences camera pose estimated pose pose error gradient R12 , t12 R 12 , t 12 R err ground truth w/o uncertainty with uncertainty Figure 1. We present a differentiable nonlinear least squares (DNLS) framework for learning feature correspondence quality by computing per-feature positional uncertainty. The uncertainty estimates (left, bottom images) are regressed from a pose estimation error (middle), enabling the framework across a range of (handcrafted, learned) feature extractors. Our learned covariances (right, orange trajectory) improve orientation estimation by up to 11% over state-of-the-art probabilistic pose estimation methods on the KITTI dataset [21].