UPS: Unified Projection Sharing for Lightweight Single-Image Super-resolution and Beyond

To date, Transformer-based frameworks have demonstrated impressive results in single-image super-resolution (SISR). However, under practical lightweight scenarios, the complex interaction of deep image feature extraction and similarity modeling limits the performance of these methods, since they require simultaneous layer-specific optimization of both two tasks. In this work, we introduce a novel Unified Projection Sharing (UPS) algorithm to decouple the feature extraction and similarity modeling. To achieve this, we establish a unified projection space defined by a learnable projection matrix, for similarity calculation across all self-attention layers. As a result, deep image feature extraction remains a per-layer optimization manner, while similarity modeling is carried out by projecting these image features onto the shared projection space. Extensive experiments demonstrate that our proposed UPS achieves state-of-the-art performance relative to leading lightweight SISR methods, as verified by various popular benchmarks. Moreover, our unified optimized projection space exhibits encouraging robustness performance for unseen data (degraded and depth images). Finally, UPS also demonstrates promising results across various image restoration tasks, including real-world and classic SISR, image denoising, and image deblocking. * Project leader † Co-first author ‡ Corresponding author 4 The dimensions remain consistent across all projection layers in SwinIR-light. Thus we can directly evaluate the pair-wise similarity scores. 38th Conference on Neural Information Processing Systems (NeurIPS 2024).