Mirror3D: Depth Refinement for Mirror Surfaces

Input image + mirror mask Raw depth Original point cloud Refined depth Refined point cloud Figure 1: We present the task of 3D mirror plane prediction and depth refinement. First, we annotate several popular RGBD datasets (Matterport3D [6], ScanNet [7], NYUv2 [32]) with 3D mirror planes. Our benchmarks show that both existing RGBD dataset 'ground truth' raw depth data, and state-of-the-art depth estimation and depth completion methods exhibit dramatic errors on mirror surfaces. We propose an architecture for 3D mirror plane estimation that refines depth estimates and produces more reliable reconstructions (compare left and right depth and point cloud pairs from NYUv2 [32] dataset).