Bidirectional Shadow Rendering for Interactive Mixed 360° Videos

In this paper, we provide a bidirectional shadow rendering method to render shadows between real and virtual objects in the 360° videos in real time. We construct a 3D scene approximation from the current output viewpoint to approximate the real scene geometry nearby in the video. Then, we propose a ray casting based algorithm to determine the shadow regions on the virtual objects cast by the real objects. After that, we introduce an object-aware shadow mapping method to cast shadows from virtual objects to real objects. Finally, we use a shadow intensity estimation algorithm to determine the shadow intensity of virtual objects and real objects to obtain shadows consistent with the input 360° video. The experiment results prove the effectiveness of our bidirectional shadow rendering method for mixed 360° videos. Our method can generate visually realistic shadows for virtual objects and real objects in 360° video in realtime, and make virtual objects more natural to integrate with real scenes in 360° videos of the mixed reality applications.