Deep Fractional Fourier Transform

Existing deep learning-based computer vision methods usually operate in the spatial and frequency domains, which are two orthogonal individual perspectives for image processing. In this paper, we introduce a new spatial-frequency analysis tool, Fractional Fourier Transform (FRFT), to provide comprehensive unified spatial-frequency perspectives. The FRFT is a unified continuous spatial-frequency transform that simultaneously reflects an image's spatial and frequency representations, making it optimal for processing non-stationary image signals. We explore the properties of the FRFT for image processing and present a fast implementation of the 2D FRFT, which facilitates its widespread use. Based on these explorations, we introduce a simple yet effective operator, Multi-order FRactional Fourier Convolution (MFRFC), which exhibits the remarkable merits of processing images from more perspectives in the spatial-frequency plane. Our proposed MFRFC is a general and basic operator that can be easily integrated into various tasks for performance improvement. We experimentally evaluate the MFRFC on various computer vision tasks, including object detection, image classification, guided super-resolution, denoising, dehazing, deraining, and low-light enhancement. Our proposed MFRFC consistently outperforms baseline methods by significant margins across all tasks. Our code is released publicly at https://github.com/yuhuUSTC/FRFT . * Equal Contribution. †Work partially performed during internship at Alibaba DAMO Academy. ‡Corresponding Author. 1 Also, a time-frequency analysis tool for time-varying signals such as video, speech, and radar signals. 37th Conference on Neural Information Processing Systems (NeurIPS 2023).