CollabVr: Reprojection-Based Edge-Client Collaborative Rendering for Real-Time High-Quality Mobile Virtual Reality

Collaborative mobile virtual reality (VR) has recently emerged as a promising solution to provide an immersive user experience with low motion-to-photo (MTP) latency. The rendering tasks are usually divided into background and foreground ones, which are executed in the edge server and head-mounted display (HMD), respectively. Assuming that the background images are static, they can be reused for temporal redundancy reduction in transmission. However, in high dynamic high-quality scenes, background images are continuously changing, making the temporal reuse strategy ineffective, leading to high MTP latency and hence motion sickness. In this paper, we propose CollabVR, a reprojection-based edge-client collaborative rendering approach for real-time high-quality mobile VR in high dynamic scenes. The key idea is to reduce the spatial redundancy in transmission by exploiting the high similarity between the left view and right view. With CollabVR, only one view is rendered and encoded in the edge server and then be transmitted to, and decoded at the HMD. The another view is reprojected by utilizing depth image-based rendering (DIBR) in the HMD, thereby greatly reducing the MTP latency even in high dynamic scenes. Furthermore, we propose a foveated-based multi-level patch subdivision strategy to achieve real-time reprojection in the resourced-limited HMD. A parallel streaming strategy is also proposed to fill holes that exist in the reprojected image. Experiments we conducted using Commercial Off-The-Shelf (COTS) devices indicate that CollabVR can reduce the average MTP latency by up to 36% compared to the baseline methods.