Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs

Many applications of Large Language Models (LLMs) require them to either simulate people or offer personalized functionality, making the demographic representativeness of LLMs crucial for equitable utility. At the same time, we know little about the extent to which these models actually reflect the demographic attributes and behaviors of certain groups or populations, with conflicting findings in empirical research. To shed light on this debate, we review 211 papers on the demographic representativeness of LLMs. We find that while 29% of the studies report positive conclusions on the representativeness of LLMs, 30% of these do not evaluate LLMs across multiple demographic categories or within demographic subcategories. Another 35% and 47% of the papers concluding positively fail to specify these subcategories altogether for gender and race, respectively. Of the articles that do report subcategories, fewer than half include marginalized groups in their study. Finally, more than a third of the papers do not define the target population to whom their findings apply; of those that do define it either implicitly or explicitly, a large majority study only the U.S. Taken together, our findings suggest an inflated perception of LLM representativeness in the broader community. We recommend more precise evaluation methods and comprehensive documentation of demographic attributes to ensure the responsible use of LLMs for social applications.

• Outsized focus on the U.S.

• 42% study the U.S.

• 36% donʼt define a clear target population • Conflicting conclusions on the representativeness of LLMs • • 29% say yes • 31% say no • Marginalized groups overlooked • Gender: 22% include non-binary gender groups • Race: 35% include diverse racial groups • Even fewer in papers claiming LLMs to be representative: 11% for gender, 24% for race Demographic Groups and Subcategories Impersonation, i.e., the LLM acts like group X Personalization, i.e., the LLM caters to or about group X Gender: women, men, nonbinary, … Ethnicity: White, Black, Asian, … Age: young, old, millennial, … Class: lower, middle, upper, … … U.S., Global, Other, Undefined A) Dimensions we annotate (N = 211) Yes, Partial, No B) Findings Advice Content Analysis "I am X. Which college "Label the toxicity of the major should I go for?" text: Abortion is murder!" Simulation Writing "As identity X. who "Write a reference letter would you vote for?" about X." Application Contexts Across groups: 0.7 for gender, 0.72 for race, … Within subcategories of a group: 0.84 for women, 0.92 for men, 0.63 for nonbinary, … None: 0.88 overall Analysis Level Conclusion on LLMsʼ Representativeness Target