Lune

CVPR2022Top-tier venue

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy

Abstract

A recurrent structure is a popular framework choice for the task of video super-resolution. The state-of-the-art method BasicVSR adopts bidirectional propagation with feature alignment to effectively exploit information from the entire input video. In this study, we redesign BasicVsr by proposing second-order grid propagation and flow-guided deformable alignment. We show that by empowering the re-current framework with enhanced propagation and align-ment, one can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a simi-lar computational constraint. In particular, our model Ba-sicVSR++ surpasses BasicVSR by a significant 0.82 dB in PSNR with similar number of parameters. BasicVSR++ is generalizable to other video restoration tasks, and obtains three champions and one first runner-up in NTIRE 2021 video restoration challenge.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

157 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • ProPainter: Improving Propagation and Transformer for Video Inpainting cites BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
  • ProPainter: Improving Propagation and Transformer for Video Inpainting cites Towards An End-to-End Framework for Flow-Guided Video Inpainting
  • ProPainter: Improving Propagation and Transformer for Video Inpainting cites Spatio-Temporal Filter Adaptive Network for Video Deblurring
  • VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation cites BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
  • Rethinking Alignment in Video Super-Resolution Transformers cites TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
  • Rethinking Alignment in Video Super-Resolution Transformers cites Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations
  • Rethinking Alignment in Video Super-Resolution Transformers cites Understanding Deformable Alignment in Video Super-Resolution
  • Rethinking Alignment in Video Super-Resolution Transformers cites BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
  • Rethinking Alignment in Video Super-Resolution Transformers cites Recurrent Video Restoration Transformer with Guided Deformable Attention
  • Understanding Deformable Alignment in Video Super-Resolution cites TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
  • Understanding Deformable Alignment in Video Super-Resolution cites Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations
  • BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond cites TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
  • BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond cites Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations
  • BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond cites Understanding Deformable Alignment in Video Super-Resolution
  • Recurrent Video Restoration Transformer with Guided Deformable Attention cites TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
  • Recurrent Video Restoration Transformer with Guided Deformable Attention cites Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations
  • Recurrent Video Restoration Transformer with Guided Deformable Attention cites Understanding Deformable Alignment in Video Super-Resolution
  • Recurrent Video Restoration Transformer with Guided Deformable Attention cites BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
  • Recurrent Video Restoration Transformer with Guided Deformable Attention cites Spatio-Temporal Filter Adaptive Network for Video Deblurring
  • Towards An End-to-End Framework for Flow-Guided Video Inpainting cites TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
  • Towards An End-to-End Framework for Flow-Guided Video Inpainting cites Understanding Deformable Alignment in Video Super-Resolution
  • Towards An End-to-End Framework for Flow-Guided Video Inpainting cites BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers157

  1. Recurrent Video Restoration Transformer with Guided Deformable AttentionJingyun Liang, Yuchen Fan, Xiaoyu Xiang, Rakesh Ranjan et al.NeurIPS 2022 · 318 citations
  2. ProPainter: Improving Propagation and Transformer for Video InpaintingShangchen Zhou, Chongyi Li, Kelvin C. K. Chan, Chen Change LoyICCV 2023 · 205 citations
  3. Towards An End-to-End Framework for Flow-Guided Video InpaintingZhen Li, Chengze Lu, Jianhua Qin, Chun-Le Guo et al.CVPR 2022 · 136 citations
  4. Rethinking Alignment in Video Super-Resolution TransformersShuwei Shi, Jinjin Gu, Liangbin Xie, Xintao Wang et al.NeurIPS 2022 · 134 citations
  5. VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow EstimationXiaoyu Shi, Zhaoyang Huang, Weikang Bian, Dasong Li et al.ICCV 2023 · 112 citations

Builds on8

  1. Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal CorrelationsPeng Yi, Zhongyuan Wang, Kui Jiang, Junjun Jiang et al.ICCV 2019 · 309 citations
  2. Spatio-Temporal Filter Adaptive Network for Video DeblurringShangchen Zhou, Jiawei Zhang, Jinshan Pan, Wangmeng Zuo et al.ICCV 2019 · 225 citations
  3. Understanding Deformable Alignment in Video Super-ResolutionKelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong et al.AAAI 2021 · 184 citations
  4. TDAN: Temporally-Deformable Alignment Network for Video Super-ResolutionYapeng Tian, Yulun Zhang, Yun Fu, Chenliang XuCVPR 2020
  5. BasicVSR: The Search for Essential Components in Video Super-Resolution and BeyondKelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong et al.CVPR 2021

Related papers

  1. ICNet: Joint Alignment and Reconstruction via Iterative Collaboration for Video Super-ResolutionJiaxu Leng, Jia Wang, Xinbo Gao, Bo Hu et al.ACM MM 2022 · 3 citations
  2. You Only Align Once: Bidirectional Interaction for Spatial-Temporal Video Super-ResolutionMengshun Hu, Kui Jiang, Zhixiang Nie, Zheng WangACM MM 2022 · 18 citations
  3. Mitigating Artifacts in Real-World Video Super-resolution ModelsLiangbin Xie, Xintao Wang, Shuwei Shi, Jinjin Gu et al.AAAI 2023 · 42 citations
  4. Exploiting Blurry Representations for Event-guided Video Super-ResolutionZeyu Xiao, Xinchao WangAAAI 2026
  5. SkipVSR: Adaptive Patch Routing for Video Super-Resolution with Inter-Frame MaskZekun Ai, Xiaotong Luo, Yanyun Qu, Yuan XieACM MM 2024 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines