TBSR: Tile-Based 360° Video Streaming with Super-Resolution on Commodity Mobile Devices

Streaming 360° videos demands excessive bandwidth. Tile-based streaming and super-resolution are two widely studied approaches to alleviate bandwidth shortage and enhance user experience in such real-time video streaming systems. The former prioritizes the transmission of a fraction of the 360° video according to the user viewport, while the latter enhances the streamed video in higher resolutions through computations. However, these two approaches bring substantial complexity and computation overhead and thus suffer from resource bottlenecks due to the constrained mobile hardware. This paper proposes TBSR, a practical mobile 360° video streaming system that incorporates in-time super-resolution with tile-based streaming on commodity mobile devices. We present the designs of three key mechanisms, including a rate adaptation method with macro tile grouping to reduce decoding computations, a decoding and SR scheduler for different types of tasks to achieve the best cost efficiency, and the workload adjustment method to control the amount of tasks given the available capabilities. We further implement the TBSR prototype. Our performance evaluation shows that TBSR outperforms the existing methods, improving QoE quality by up to 32% and bandwidth savings by 26%.