GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding

Figure 1. Our method, called GP-NeRF, achieves remarkable performance improvements for instance and semantic segmentation in both synthesis [35] and real-world [10] datasets, as shown in the right column of the figure. Here we showcase generalized semantic segmentation, finetuning semantic segmentation, and instance segmentation) with their corresponding reconstruction results. For the left column, the qualitative results of the visualization are presented, showing the effectiveness of our method for simultaneous segmentation and reconstruction. What's more, we visualize our rendered features via PCA in the novel view, demonstrating our method possesses the capability to produce semantic-aware features that can distinguish between different classes and objects.