GAN-Control: Explicitly Controllable GANs
Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gérard G. Medioni
Abstract
We present a framework for training GANs with explicit control over generated facial images. We are able to control the generated image by settings exact attributes such as age, pose, expression, etc. Most approaches for manipulating GAN-generated images achieve partial control by leveraging the latent space disentanglement properties, obtained implicitly after standard GAN training. Such methods are able to change the relative intensity of certain attributes, but not explicitly set their values. Recently proposed methods, designed for explicit control over human faces, harness morphable 3D face models (3DMM) to allow fine-grained control capabilities in GANs. Unlike these methods, our control is not constrained to 3DMM parameters and is extendable beyond the domain of human faces. Using contrastive learning, we obtain GANs with an explicitly disentangled latent space. This disentanglement is utilized to train control-encoders mapping human-interpretable inputs to suitable latent vectors, thus allowing explicit control. In the domain of human faces we demonstrate control over identity, age, pose, expression, hair color and illumination. We also demonstrate control capabilities of our framework in the domains of painted portraits and dog image generation. We demonstrate that our approach achieves state-of-the-art performance both qualitatively and quantitatively.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 34d50dfe-0c00-4fc7-baa3-cb535b0042a6Cited by top-tier papers27
- StyleSDF: High-Resolution 3D-Consistent Image and Geometry GenerationRoy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman et al.CVPR 2022 · 229 citations
- DiffEdit: Diffusion-based semantic image editing with mask guidanceGuillaume Couairon, Jakob Verbeek, Holger Schwenk, Matthieu CordICLR 2023 · 102 citations
- SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and EditingYichun Shi, Xiao Yang, Yangyue Wan, Xiaohui ShenCVPR 2022 · 88 citations
- OBJECT 3DIT: Language-guided 3D-aware Image EditingOscar Michel, Anand Bhattad, Eli VanderBilt, Ranjay Krishna et al.NeurIPS 2023 · 79 citations
- Controllable 3D Face Synthesis with Conditional Generative Occupancy FieldsKeqiang Sun, Shangzhe Wu, Zhaoyang Huang, Ning Zhang et al.NeurIPS 2022 · 62 citations
Builds on14
- Training Generative Adversarial Networks with Limited DataTero Karras, Miika Aittala, Janne Hellsten, Samuli Laine et al.NeurIPS 2020 · 2,345 citations
- Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?Rameen Abdal, Yipeng Qin, Peter WonkaICCV 2019 · 1,195 citations
- GANSpace: Discovering Interpretable GAN ControlsErik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain ParisNeurIPS 2020 · 1,049 citations
- On the "steerability" of generative adversarial networksAli Jahanian, Lucy Chai, Phillip IsolaICLR 2020 · 421 citations
- Efficient Facial Feature Learning with Wide Ensemble-Based Convolutional Neural NetworksHenrique Siqueira, Sven Magg, Stefan WermterAAAI 2020 · 136 citations
Related papers
- MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image ManipulationSafa C. Medin, Bernhard Egger, Anoop Cherian, Ye Wang et al.AAAI 2022 · 38 citations
- StyleRig: Rigging StyleGAN for 3D Control Over Portrait ImagesAyush Tewari, Mohamed A. Elgharib, Gaurav Bharaj, Florian Bernard et al.CVPR 2020
- Conceptual and Hierarchical Latent Space Decomposition for Face EditingSavas Özkan, Mete Özay, Tom RobinsonICCV 2023 · 3 citations
- A 3D GAN for Improved Large-Pose Facial RecognitionRichard T. Marriott, Sami Romdhani, Liming ChenCVPR 2021
- XAGen: 3D Expressive Human Avatars GenerationZhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Jiashi Feng et al.NeurIPS 2023 · 24 citations
