DINN360: Deformable Invertible Neural Network for Latitude-aware 360° Image Rescaling

With the rapid development of virtual reality, 360 • images have gained increasing popularity. Their wide field of view necessitates high resolution to ensure image quality. This, however, makes it harder to acquire, store and even process such 360 • images. To alleviate this issue, we propose the first attempt at 360 • image rescaling, which refers to downscaling a 360 • image to a visually valid lowresolution (LR) counterpart and then upscaling to a highresolution (HR) 360 • image given the LR variant. Specifically, we first analyze two 360 • image datasets and observe several findings that characterize how 360 • images typically change along their latitudes. Inspired by these findings, we propose a novel deformable invertible neural network (INN), named DINN360, for latitude-aware 360 • image rescaling. In DINN360, a deformable INN is designed to downscale the LR image, and project the high-frequency (HF) component to the latent space by adaptively handling various deformations occurring at different latitude regions. Given the downscaled LR image, the high-quality HR image is then reconstructed in a conditional latitude-aware manner by recovering the structure-related HF component from the latent space. Extensive experiments over four public datasets show that our DINN360 method performs considerably better than other state-of-the-art methods for 2×, 4× and 8× 360 • image rescaling.