Zero-Shot 4D Lidar Panoptic Segmentation

Prior methods (left) for zero-shot Lidar panoptic segmentation process individual (3D) point clouds in isolation. In contrast, our data-driven approach (right) operates directly on sequences of point clouds, jointly performing object segmentation, tracking, and zero-shot recognition based on text prompts specified at test time. Our method localizes and tracks any object and provides a temporally coherent semantic interpretation of dynamic scenes. We can correctly segment canonical objects, such as car, and objects beyond the vocabularies of standard Lidar datasets, such as advertising stand. Best seen in color, zoomed.