InsetGAN for Full-Body Image Generation
Anna Frühstück, Krishna Kumar Singh, Eli Shechtman, Niloy J. Mitra, Peter Wonka, Jingwan Lu
Abstract
While GANs can produce photo-realistic images in ideal conditions for certain domains, the generation of full-body human images remains difficult due to the diversity of identities, hairstyles, clothing, and the variance in pose. In-stead of modeling this complex domain with a single GAN, we propose a novel method to combine multiple pretrained GANs, where one GAN generates a global canvas (e.g., human body) and a set of specialized GANs, or insets, focus on different parts (e.g., faces, shoes) that can be seamlessly inserted onto the global canvas. We model the problem as jointly exploring the respective latent spaces such that the generated images can be combined, by inserting the parts from the specialized generators onto the global canvas, without introducing seams. We demonstrate the setup by combining a full body GAN with a dedicated high-quality face GAN to produce plausible-looking humans. We evalu-ate our results with quantitative metrics and user studies.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 7ed8ed7a-b6c9-421a-a2c6-51fe06677baeCited by top-tier papers21
- Generative Neural Articulated Radiance FieldsAlexander W. Bergman, Petr Kellnhofer, Wang Yifan, Eric R. Chan et al.NeurIPS 2022 · 144 citations
- Text2Human: text-driven controllable human image generationYuming Jiang, Shuai Yang, Haonan Qiu, Wayne Wu et al.SIGGRAPH 2022 · 140 citations
- PrimDiffusion: Volumetric Primitives Diffusion for 3D Human GenerationZhaoxi Chen, Fangzhou Hong, Haiyi Mei, Guangcong Wang et al.NeurIPS 2023 · 44 citations
- DINAR: Diffusion Inpainting of Neural Textures for One-Shot Human AvatarsDavid Svitov, Dmitrii Gudkov, Renat Bashirov, Victor LempitskyICCV 2023 · 37 citations
- EVA3D: Compositional 3D Human Generation from 2D Image CollectionsFangzhou Hong, Zhaoxi Chen, Yushi Lan, Liang Pan et al.ICLR 2023 · 35 citations
Builds on15
- Training Generative Adversarial Networks with Limited DataTero Karras, Miika Aittala, Janne Hellsten, Samuli Laine et al.NeurIPS 2020 · 2,345 citations
- Alias-Free Generative Adversarial NetworksTero Karras, Miika Aittala, Samuli Laine, Erik Härkönen et al.NeurIPS 2021 · 2,126 citations
- Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?Rameen Abdal, Yipeng Qin, Peter WonkaICCV 2019 · 1,195 citations
- ReStyle: A Residual-Based StyleGAN Encoder via Iterative RefinementYuval Alaluf, Or Patashnik, Daniel Cohen-OrICCV 2021 · 377 citations
- Large Scale Image Completion via Co-Modulated Generative Adversarial NetworksShengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong et al.ICLR 2021 · 348 citations
Related papers
- BodyGAN: General-purpose Controllable Neural Human Body GenerationChaojie Yang, Hanhui Li, Shengjie Wu, Shengkai Zhang et al.CVPR 2022 · 8 citations
- 3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose MappingZhuoqian Yang, Shikai Li, Wayne Wu, Bo DaiICCV 2023 · 19 citations
- Controllable Person Image Synthesis With Attribute-Decomposed GANYifang Men, Yiming Mao, Yuning Jiang, Wei-Ying Ma et al.CVPR 2020
- Aggregated Multi-GANs for Controlled 3D Human Motion PredictionZhenguang Liu, Kedi Lyu, Shuang Wu, Haipeng Chen et al.AAAI 2021 · 64 citations
- Wish You Were Here: Context-Aware Human GenerationOran Gafni, Lior WolfCVPR 2020
