Lune

AAAI2024Top-tier venue

Limitations of Face Image Generation

Harrison Rosenberg, Shimaa Ahmed, Guruprasad V. Ramesh, Kassem Fawaz, Ramya Korlakai Vinayak

Abstract

Text-to-image diffusion models have achieved widespread popularity due to their unprecedented image generation capability. In particular, their ability to synthesize and modify human faces has spurred research into using generated face images in both training data augmentation and model performance assessments. In this paper, we study the efficacy and shortcomings of generative models in the context of face generation. Utilizing a combination of qualitative and quantitative measures, including embedding-based metrics and user studies, we present a framework to audit the characteristics of generated faces conditioned on a set of social attributes. We applied our framework on faces generated through stateof-the-art text-to-image diffusion models. We identify several limitations of face image generation that include faithfulness to the text prompt, demographic disparities, and distributional shifts. Furthermore, we present an analytical model that provides insights into how training data selection contributes to the performance of generative models. Our survey data and analytics code can be found online 2 . 2 https://github.com/wi-pi/Limitations_of_Face_Generation

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • High-Resolution Image Synthesis with Latent Diffusion Models cites Zero-Shot Text-to-Image Generation
  • Zero-Shot Text-to-Image Generation cites Learning Transferable Visual Models From Natural Language Supervision
  • StyleGAN-NADA: CLIP-guided domain adaptation of image generators cites Learning Transferable Visual Models From Natural Language Supervision
  • MAUGen: A Unified Diffusion Approach for Multi-Identity Facial Expression and AU Label Generation cites High-Resolution Image Synthesis with Latent Diffusion Models
  • MAUGen: A Unified Diffusion Approach for Multi-Identity Facial Expression and AU Label Generation cites Learning Transferable Visual Models From Natural Language Supervision
  • MAUGen: A Unified Diffusion Approach for Multi-Identity Facial Expression and AU Label Generation cites StyleGAN-NADA: CLIP-guided domain adaptation of image generators
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. MAUGen: A Unified Diffusion Approach for Multi-Identity Facial Expression and AU Label GenerationXiangdong Li, Ye Lou, Ao Gao, Wei Zhang et al.AAAI 2026

Builds on9

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  3. Zero-Shot Text-to-Image GenerationAditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray et al.ICML 2021 · 6,356 citations
  4. Reliable Fidelity and Diversity Metrics for Generative ModelsMuhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi et al.ICML 2020 · 553 citations
  5. StyleGAN-NADA: CLIP-guided domain adaptation of image generatorsRinon Gal, Or Patashnik, Haggai Maron, Amit H. Bermano et al.SIGGRAPH 2022 · 501 citations

Related papers

  1. Balancing Act: Distribution-Guided Debiasing in Diffusion ModelsRishubh Parihar, Abhijnya Bhat, Abhipsa Basu, Saswat Mallick et al.CVPR 2024 · 15 citations
  2. Fair Text-to-Image Diffusion via Fair MappingJia Li, Lijie Hu, Jingfeng Zhang, Tianhang Zheng et al.AAAI 2025 · 36 citations
  3. ZeroFake: Zero-Shot Detection of Fake Images Generated and Edited by Text-to-Image Generation ModelsZeyang Sha, Yicong Tan, Mingjie Li, Michael Backes et al.CCS 2024 · 8 citations
  4. FairGen: Controlling Sensitive Attributes for Fair Generations in Diffusion Models via Adaptive Latent GuidanceMintong Kang, Vinayshekhar Bannihatti Kumar, Shamik Roy, Abhishek Kumar et al.EMNLP 2025
  5. FairImagen: Post-Processing for Bias Mitigation in Text-to-Image ModelsZihao Fu, Ryan Brown, Shun Shao, Kai Rawal et al.NeurIPS 2025 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines