An Analytical Solution to Gauss-Newton Loss for Direct Image Alignment

Figure 1: Direct image alignment is a technique for aligning scenes based on image intensities. Recent learning-based methods seek to improve its success rate by increasing the convergence basin. We derive an analytical solution to the core idea of such methods, the Gauss-Newton loss, enabling fine-grained control over the basin of convergence. As a result, we can successfully align two scenes despite a highly imprecise initialization. From left to right, the example above illustrates the convergence of the reprojected keypoints (red) to the ground truth (green) by optimizing the SE(3) camera pose with our analytical solution. The blue color is the re-projection in the first iteration.