Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression

Content-adaptive compression is crucial for enhancing the adaptability of the pre-trained neural codec for various contents. Though, its application in neural video compression (NVC) is still limited due to two main aspects: 1), video compression relies heavily on temporal redundancy, therefore updating just one or a few frames can lead to significant errors accumulating over time; 2), NVC frameworks are generally complex, with many comprehensive components that are not trivial to update quickly during the encoding procedure. To address these challenges, we have developed a content-adaptive NVC technique called Group-aware Parameter-efficient Updating (GPU). Initially, to minimize error accumulation, we adopt a group-aware approach for updating encoder parameters. This involves adopting a patch-based Group of Pictures (GoP) updating strategy to segment a video into patch-based GoPs, which will be updated to facilitate a globally optimized domain-transferable solution. Subsequently, we introduce a parameter-efficient delta-tuning strategy, which is achieved by integrating several light-weight adapters into each encoding component by using both serial and parallel configuration. Such architecture-agnostic modules stimulate the components with large parameters, thereby reducing the updating cost during the encoding stage. We incorporate our GPU into the latest NVC framework and conduct extensive experiments, whose results showcase outstanding video compression efficiency across six video compression benchmarks and the adaptability of one medical volumetric image compression benchmark.