Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences

Figure 1 . Our method learns to see behind objects in RGB-D sequences in order to achieve robust dynamic object tracking; we predict the complete underlying geometry of each object beyond the observed view, which enables finding correspondences which can more reliably persist over time, under various view changes and object motion. From an input RGB-D frame, we first perform 3D object detection, then jointly infer for each object its complete geometry and dense correspondence mapping to its canonical space. These correspondences on the predicted complete object geometry help to provide robust multi-object tracking over time.