Feature Proliferation - the "Cancer" in StyleGAN and its Treatments
Shuang Song, Yuanbang Liang, Jing Wu, Yu-Kun Lai, Yipeng Qin
Abstract
Despite the success of StyleGAN in image synthesis, the images it synthesizes are not always perfect and the well-known truncation trick has become a standard postprocessing technique for StyleGAN to synthesize high quality images. Although effective, it has long been noted that the truncation trick tends to reduce the diversity of synthesized images and unnecessarily sacrifices many distinct image features. To address this issue, in this paper, we first delve into the StyleGAN image synthesis mechanism and discover an important phenomenon, namely Feature Proliferation, which demonstrates how specific features reproduce with forward propagation. Then, we show how the occurrence of Feature Proliferation results in StyleGAN image artifacts. As an analogy, we refer to it as the "cancer" in StyleGAN from its proliferating and malignant nature. Finally, we propose a novel feature rescaling method that identifies and modulates risky features to mitigate feature proliferation. Thanks to our discovery of Feature Proliferation, the proposed feature rescaling method is less destructive and retains more useful image features than the truncation trick, as it is more fine-grained and works in a lower-level feature space rather than a high-level latent space. Experimental results justify the validity of our claims and the effectiveness of the proposed feature rescaling method. Our code is available at https:// github.com/ songc42/ Feature-proliferation.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis cites Alias-Free Generative Adversarial Networks
- HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing cites Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
- HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing cites Alias-Free Generative Adversarial Networks
- HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing cites Training Generative Adversarial Networks with Limited Data
- Alias-Free Generative Adversarial Networks cites Training Generative Adversarial Networks with Limited Data
Builds on11
- Training Generative Adversarial Networks with Limited DataTero Karras, Miika Aittala, Janne Hellsten, Samuli Laine et al.NeurIPS 2020 · 2,345 citations
- Alias-Free Generative Adversarial NetworksTero Karras, Miika Aittala, Samuli Laine, Erik Härkönen et al.NeurIPS 2021 · 2,126 citations
- Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?Rameen Abdal, Yipeng Qin, Peter WonkaICCV 2019 · 1,195 citations
- StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image SynthesisAxel Sauer, Tero Karras, Samuli Laine, Andreas Geiger et al.ICML 2023 · 284 citations
- HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image EditingYuval Alaluf, Omer Tov, Ron Mokady, Rinon Gal et al.CVPR 2022 · 250 citations
Related papers
- Analyzing and Improving the Image Quality of StyleGANTero Karras, Samuli Laine, Miika Aittala, Janne Hellsten et al.CVPR 2020
- StyleRes: Transforming the Residuals for Real Image Editing with StyleGANHamza Pehlivan, Yusuf Dalva, Aysegul DundarCVPR 2023
- Self-Distilled StyleGAN: Towards Generation from Internet PhotosRon Mokady, Omer Tov, Michal Yarom, Oran Lang et al.SIGGRAPH 2022 · 29 citations
- StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned FacesShuai Yang, Liming Jiang, Ziwei Liu, Chen Change LoyICCV 2023 · 33 citations
- StyleCariGAN: caricature generation via StyleGAN feature map modulationWonjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang et al.SIGGRAPH 2021 · 51 citations
