Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
Liming Jiang, Bo Dai, Wayne Wu, Chen Change Loy
Abstract
Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting, the underlying cause that impedes the generator's convergence. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator. As an alternative method to existing approaches that rely on standard data augmentations or model regularization, APA alleviates overfitting by employing the generator itself to augment the real data distribution with generated images, which deceives the discriminator adaptively. Extensive experiments demonstrate the effectiveness of APA in improving synthesis quality in the low-data regime. We provide a theoretical analysis to examine the convergence and rationality of our new training strategy. APA is simple and effective. It can be added seamlessly to powerful contemporary GANs, such as StyleGAN2, with negligible computational cost. Code: https://github.com/EndlessSora/DeceiveD .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
27 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Differentiable Augmentation for Data-Efficient GAN Training cites Consistency Regularization for Generative Adversarial Networks
- Domain Re-Modulation for Few-Shot Generative Domain Adaptation cites Differentiable Augmentation for Data-Efficient GAN Training
- Domain Re-Modulation for Few-Shot Generative Domain Adaptation cites Training Generative Adversarial Networks with Limited Data
- StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery cites Training Generative Adversarial Networks with Limited Data
- Masked Generative Adversarial Networks are Data-Efficient Generation Learners cites Differentiable Augmentation for Data-Efficient GAN Training
- Masked Generative Adversarial Networks are Data-Efficient Generation Learners cites Training Generative Adversarial Networks with Limited Data
- FreGAN: Exploiting Frequency Components for Training GANs under Limited Data cites Differentiable Augmentation for Data-Efficient GAN Training
- FreGAN: Exploiting Frequency Components for Training GANs under Limited Data cites Training Generative Adversarial Networks with Limited Data
- Training Generative Adversarial Networks with Limited Data cites Differentiable Augmentation for Data-Efficient GAN Training
- Training Generative Adversarial Networks with Limited Data cites Consistency Regularization for Generative Adversarial Networks
- Improving GANs with A Dynamic Discriminator cites Differentiable Augmentation for Data-Efficient GAN Training
- Improving GANs with A Dynamic Discriminator cites Consistency Regularization for Generative Adversarial Networks
- Improving GANs with A Dynamic Discriminator cites Training Generative Adversarial Networks with Limited Data
Cited by top-tier papers27
- Pastiche Master: Exemplar-Based High-Resolution Portrait Style TransferShuai Yang, Liming Jiang, Ziwei Liu, Chen Change LoyCVPR 2022 · 130 citations
- FreGAN: Exploiting Frequency Components for Training GANs under Limited DataMengping Yang, Zhe Wang, Ziqiu Chi, Yanbing ZhangNeurIPS 2022 · 49 citations
- Improving GANs with A Dynamic DiscriminatorCeyuan Yang, Yujun Shen, Yinghao Xu, Deli Zhao et al.NeurIPS 2022 · 42 citations
- Masked Generative Adversarial Networks are Data-Efficient Generation LearnersJiaxing Huang, Kaiwen Cui, Dayan Guan, Aoran Xiao et al.NeurIPS 2022 · 33 citations
- Domain Re-Modulation for Few-Shot Generative Domain AdaptationYi Wu, Ziqiang Li, Chaoyue Wang, Heliang Zheng et al.NeurIPS 2023 · 31 citations
Builds on11
- Training Generative Adversarial Networks with Limited DataTero Karras, Miika Aittala, Janne Hellsten, Samuli Laine et al.NeurIPS 2020 · 2,345 citations
- StyleCLIP: Text-Driven Manipulation of StyleGAN ImageryOr Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or et al.ICCV 2021 · 1,437 citations
- Differentiable Augmentation for Data-Efficient GAN TrainingShengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu et al.NeurIPS 2020 · 707 citations
- Focal Frequency Loss for Image Reconstruction and SynthesisLiming Jiang, Bo Dai, Wayne Wu, Chen Change LoyICCV 2021 · 422 citations
- Consistency Regularization for Generative Adversarial NetworksHan Zhang, Zizhao Zhang, Augustus Odena, Honglak LeeICLR 2020 · 305 citations
Related papers
- Improving the Training of the GANs with Limited Data via Dual Adaptive Noise InjectionZhaoyu Zhang, Yang Hua, Guanxiong Sun, Hui Wang et al.ACM MM 2024 · 3 citations
- Augmentation-Aware Self-Supervision for Data-Efficient GAN TrainingLiang Hou, Qi Cao, Yige Yuan, Songtao Zhao et al.NeurIPS 2023 · 15 citations
- DigGAN: Discriminator gradIent Gap Regularization for GAN Training with Limited DataTiantian Fang, Ruoyu Sun, Alexander G. SchwingNeurIPS 2022 · 27 citations
- Data-Efficient Instance Generation from Instance DiscriminationCeyuan Yang, Yujun Shen, Yinghao Xu, Bolei ZhouNeurIPS 2021 · 98 citations
- Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket PerspectiveTianlong Chen, Yu Cheng, Zhe Gan, Jingjing Liu et al.NeurIPS 2021 · 61 citations
