Lune

NeurIPS2024Top-tier venue

UPS: Unified Projection Sharing for Lightweight Single-Image Super-resolution and Beyond

Kun Zhou, Xinyu Lin, Zhonghang Liu, Xiaoguang Han, Jiangbo Lu

Abstract

To date, Transformer-based frameworks have demonstrated impressive results in single-image super-resolution (SISR). However, under practical lightweight scenarios, the complex interaction of deep image feature extraction and similarity modeling limits the performance of these methods, since they require simultaneous layer-specific optimization of both two tasks. In this work, we introduce a novel Unified Projection Sharing (UPS) algorithm to decouple the feature extraction and similarity modeling. To achieve this, we establish a unified projection space defined by a learnable projection matrix, for similarity calculation across all self-attention layers. As a result, deep image feature extraction remains a per-layer optimization manner, while similarity modeling is carried out by projecting these image features onto the shared projection space. Extensive experiments demonstrate that our proposed UPS achieves state-of-the-art performance relative to leading lightweight SISR methods, as verified by various popular benchmarks. Moreover, our unified optimized projection space exhibits encouraging robustness performance for unseen data (degraded and depth images). Finally, UPS also demonstrates promising results across various image restoration tasks, including real-world and classic SISR, image denoising, and image deblocking.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Restormer: Efficient Transformer for High-Resolution Image Restoration cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  • ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting cites Designing a Practical Degradation Model for Deep Blind Image Super-Resolution
  • ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  • MAXIM: Multi-Axis MLP for Image Processing cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  • Emulating Self-attention with Convolution for Efficient Image Super-Resolution cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Emulating Self-attention with Convolution for Efficient Image Super-ResolutionDongheon Lee, Seokju Yun, Youngmin RoICCV 2025 · 19 citations

Builds on16

  1. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
  2. Restormer: Efficient Transformer for High-Resolution Image RestorationSyed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat et al.CVPR 2022 · 3,348 citations
  3. Designing a Practical Degradation Model for Deep Blind Image Super-ResolutionKai Zhang, Jingyun Liang, Luc Van Gool, Radu TimofteICCV 2021 · 898 citations
  4. ResShift: Efficient Diffusion Model for Image Super-resolution by Residual ShiftingZongsheng Yue, Jianyi Wang, Chen Change LoyNeurIPS 2023 · 646 citations
  5. MAXIM: Multi-Axis MLP for Image ProcessingZhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang et al.CVPR 2022 · 550 citations

Related papers

  1. Progressive Focused Transformer for Single Image Super-ResolutionWei Long, Xingyu Zhou, Leheng Zhang, Shuhang GuCVPR 2025
  2. Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-ResolutionKaram Park, Jae Woong Soh, Nam Ik ChoAAAI 2025 · 20 citations
  3. LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-resolution and BeyondWenbo Li, Kun Zhou, Lu Qi, Nianjuan Jiang et al.NeurIPS 2020 · 293 citations
  4. Robust Reference-Based Super-Resolution With Similarity-Aware Deformable ConvolutionGyumin Shim, Jinsun Park, In So KweonCVPR 2020
  5. Compacter: A Lightweight Transformer for Image RestorationZhijian Wu, Jun Li, Yang Hu, Dingjiang HuangACM MM 2024
Dusk over the sea between two cliffs drawn in fine vertical lines