Consistent Direct Time-of-Flight Video Depth Super-Resolution

Figure 1. We propose the first multi-frame approaches, dToF depth video super-resolution (DVSR) and histogram video super-resolution (HVSR), to super-resolve low-resolution dToF sensor videos with the high-resolution RGB frame guidance. The point cloud visualizations of depth predictions reveal that, by utilizing multi-frame correlation, DVSR predicts significantly better geometry compared to state-of-theart per-frame depth enhancement networks [42] while being more lightweight; HVSR further improves the fidelity of geometry and reduces flying pixels by utilizing the dToF histogram information. Besides the improvements in per-frame estimation, we highly recommend readers to check out the supplementary video, which visualizes the significant improvements in temporal stability across the entire sequences.