Non-rigid Point Cloud Registration with Neural Deformation Pyramid

Non-rigid point cloud registration is a key component in many computer vision and computer graphics applications. The high complexity of the unknown non-rigid motion make this task a challenging problem. In this paper, we break down this problem via hierarchical motion decomposition. Our method called Neural Deformation Pyramid (NDP) represents non-rigid motion using a pyramid architecture. Each pyramid level, denoted by a Multi-Layer Perception (MLP), takes as input a sinusoidally encoded 3D point and outputs its motion increments from the previous level. The sinusoidal function starts with a low input frequency and gradually increases when the pyramid level goes down. This allows a multi-level rigid to nonrigid motion decomposition and also speeds up the solving by 50 times compared to the existing MLP-based approach. Our method achieves advanced partialto-partial non-rigid point cloud registration results on the 4DMatch/4DLoMatch benchmark under both no-learned and supervised settings. Code is available at https://github.com/rabbityl/DeformationPyramid . Non-rigid point cloud registration is a key component in many computer vision and computer graphics applications. The goal of non-rigid registration is to find the transformation that maps one point cloud to another. With the availability of consumer range sensors that can measure time-varying surface points, non-rigid registration has been applied to dynamic shape reconstruction problems such as human performance capture, enabling a wide range of applications in XR and robotics. Non-rigid registration is a challenging problem. First, 3D sensor measurements often contain noise, outliers, and occlusions. Occlusions often lead to disconnection of point cloud geometry. Point clouds may also have very low overlap ratios with each other due to the scene deformation and sensor's viewpoint change. The most challenging thing is, unlike rigid registration that only needs to determine the rotation and translation parameters, non-rigid registration needs to estimate the unknown movement of all points, making this problem especially complex to solve. In this paper, we alleviate this complexity through motion decomposition. We observe that natural non-rigid motion usually forms a hierarchical structure: with higher hierarchies representing the global movements and lower hierarchies representing the local deformation. For instance, a walking person can be roughly approximated by three levels: 1) global location and orientation change, 2) local articulated movements from arms, legs, etc, and 3) fine-grained cloth deformation caused by exterior forces. Each level represents motion at a different scale and they have top-down dependencies. Based on this observation, we propose a hierarchical motion representation called Neural Deformation Pyramid (NDP) for non-rigid registration. NDP has a pyramid architecture. Each pyramid level contains a Multi-Layer Perception (MLP) that takes as input a sinusoidally encoded 3D point and outputs its motion increments from the previous level. We found that the frequency of the sinusoidal function controls MLP's capacity of representing non-rigidity: low frequencies yield smooth signals 36th Conference on Neural Information Processing Systems (NeurIPS 2022).