Semantic Guided Part Relation-aware Network for Point Cloud Completion

The primary goal of 3D point cloud completion is to reconstruct complete and high-resolution point clouds from incomplete and low-resolution inputs. Although recent approaches have achieved satisfactory completion performance by incorporating additional images, there remains substantial room for improvement in fully harnessing the rich geometric relational information inherent in the parts. To address this challenge, we propose a novel Semantic Guided Part Relation-aware Network (SGPRNet) for Point Cloud Completion. Its core innovation lies in establishing part semantic relations to guide the reconstruction of structurally consistent local geometries. Specifically, we utilize Multi-modal Large Language Models (MLLMs) to automatically generate the specific text of 3D shapes, which provides detailed descriptions of geometric part relations. Building upon this, we design an Orthogonal Semantic Part Transfer (OSPT) module that learns transferable semantic relations between geometric parts. Subsequently, we develop a Semantic Geometric Relation-aware Transformer (SGRFormer) to progressively refine these semantic features, enhancing point cloud representations and guiding the generation of fine local structures. In addition, we establish a point-text pairs corpus, OmniObject3D-212/34 and Text-ViPC datasets based on existing OmniObject3D and ShapeNet-ViPC datasets, incorporating the specific text. Extensive experimental results demonstrate that our method outperforms existing state-ofthe-art completion methods.