Lune

ICML2020Top-tier venue

Leveraging Frequency Analysis for Deep Fake Image Recognition

Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, Thorsten Holz

Abstract

Deep neural networks can generate images that are astonishingly realistic, so much so that it is often hard for humans to distinguish them from actual photos. These achievements have been largely made possible by Generative Adversarial Networks (GANs). While deep fake images have been thoroughly investigated in the image domain - a classical approach from the area of image forensics - an analysis in the frequency domain has been missing so far. In this paper, we address this shortcoming and our results reveal that in frequency space, GAN-generated images exhibit severe artifacts that can be easily identified. We perform a comprehensive analysis, showing that these artifacts are consistent across different neural network architectures, data sets, and resolutions. In a further investigation, we demonstrate that these artifacts are caused by upsampling operations found in all current GAN architectures, indicating a structural and fundamental problem in the way images are generated via GANs. Based on this analysis, we demonstrate how the frequency representation can be used to identify deep fake images in an automated way, surpassing state-of-the-art methods.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

185 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • The Stable Signature: Rooting Watermarks in Latent Diffusion Models cites CNN-Generated Images Are Surprisingly Easy to Spot... for Now
  • The Stable Signature: Rooting Watermarks in Latent Diffusion Models cites Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
  • DIRE for Diffusion-Generated Image Detection cites CNN-Generated Images Are Surprisingly Easy to Spot... for Now
  • DIRE for Diffusion-Generated Image Detection cites Detecting Deepfakes with Self-Blended Images
  • DIRE for Diffusion-Generated Image Detection cites Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
  • Detecting Deepfakes with Self-Blended Images cites Exploring Temporal Coherence for More General Video Face Forgery Detection
  • Focal Frequency Loss for Image Reconstruction and Synthesis cites CNN-Generated Images Are Surprisingly Easy to Spot... for Now
  • Focal Frequency Loss for Image Reconstruction and Synthesis cites Watch Your Up-Convolution: CNN Based Generative Deep Neural Networks Are Failing to Reproduce Spectral Distributions
  • Exploring Temporal Coherence for More General Video Face Forgery Detection cites CNN-Generated Images Are Surprisingly Easy to Spot... for Now
  • Exploring Temporal Coherence for More General Video Face Forgery Detection cites Watch Your Up-Convolution: CNN Based Generative Deep Neural Networks Are Failing to Reproduce Spectral Distributions
  • Watch Your Up-Convolution: CNN Based Generative Deep Neural Networks Are Failing to Reproduce Spectral Distributions cites Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers185

  1. DIRE for Diffusion-Generated Image DetectionZhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang et al.ICCV 2023 · 479 citations
  2. Focal Frequency Loss for Image Reconstruction and SynthesisLiming Jiang, Bo Dai, Wayne Wu, Chen Change LoyICCV 2021 · 422 citations
  3. The Stable Signature: Rooting Watermarks in Latent Diffusion ModelsPierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze et al.ICCV 2023 · 370 citations
  4. Detecting Deepfakes with Self-Blended ImagesKaede Shiohara, Toshihiko YamasakiCVPR 2022 · 366 citations
  5. Exploring Temporal Coherence for More General Video Face Forgery DetectionYinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng et al.ICCV 2021 · 314 citations

Builds on3

  1. Attributing Fake Images to GANs: Learning and Analyzing GAN FingerprintsNing Yu, Larry Davis, Mario FritzICCV 2019 · 533 citations
  2. CNN-Generated Images Are Surprisingly Easy to Spot... for NowSheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens et al.CVPR 2020
  3. Watch Your Up-Convolution: CNN Based Generative Deep Neural Networks Are Failing to Reproduce Spectral DistributionsRicard Durall, Margret Keuper, Janis KeuperCVPR 2020

Related papers

  1. Fourier Spectrum Discrepancies in Deep Network Generated ImagesTarik Dzanic, Karan Shah, Freddie D. WitherdenNeurIPS 2020 · 235 citations
  2. Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain LearningChuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu et al.AAAI 2024 · 232 citations
  3. On the Frequency Bias of Generative ModelsKatja Schwarz, Yiyi Liao, Andreas GeigerNeurIPS 2021 · 117 citations
  4. Spectral Distribution Aware Image GenerationSteffen Jung, Margret KeuperAAAI 2021 · 42 citations
  5. Rethinking the Up-Sampling Operations in CNN-Based Generative Network for Generalizable Deepfake DetectionChuangchuang Tan, Huan Liu, Yao Zhao, Shikui Wei et al.CVPR 2024 · 126 citations
Dusk over the sea between two cliffs drawn in fine vertical lines