Simulacrum of Stories: Examining Large Language Models as Qualitative Research Participants
Shivani Kapania, William Agnew, Motahhare Eslami, Hoda Heidari, Sarah E. Fox
摘要
The recent excitement around generative models has sparked a wave of proposals suggesting the replacement of human participation and labor in research and development–e.g., through surveys, experiments, and interviews—with synthetic research data generated by large language models (LLMs). We conducted interviews with 19 qualitative researchers to understand their perspectives on this paradigm shift. Initially skeptical, researchers were surprised to see similar narratives emerge in the LLM-generated data when using the interview probe. However, over several conversational turns, they went on to identify fundamental limitations, such as how LLMs foreclose participants’ consent and agency, produce responses lacking in palpability and contextual depth, and risk delegitimizing qualitative research methods. We argue that the use of LLMs as proxies for participants enacts the surrogate effect, raising ethical and epistemological concerns that extend beyond the technical limitations of current models to the core of whether LLMs fit within qualitative ways of knowing.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper9
- Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public OpinionsJoseph Suh, Erfan Jahanparast, Suhong Moon, Minwoo Kang 等ACL 2025 · 被引用 48 次
- A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language TechnologiesAlicia DeVrio, Myra Cheng, Lisa Egede, Alexandra Olteanu 等CHI 2025 · 被引用 28 次
- Spectrum Tuning: Post-Training for Distributional Coverage and In-Context SteerabilityTaylor Sorensen, Benjamin Newman, Jared Moore, Chan Young Park 等ICLR 2026 · 被引用 19 次
- Towards Better Reflexive Thematic Analysis in HCI: A Scoping Review of Practice at CHIJacob M. Rigby, Ioanna IacovidesCHI 2026 · 被引用 6 次
- Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 ArticlesDanial Amin, Joni Salminen, Farhan Ahmed, Sonja M. H. Tervola 等CHI 2026 · 被引用 6 次
它引用的顶会 Paper28
- Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris 等UIST 2023 · 被引用 1,882 次
- AlpacaFarm: A Simulation Framework for Methods that Learn from Human FeedbackYann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang 等NeurIPS 2023 · 被引用 948 次
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent DebateChi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu 等ICLR 2024 · 被引用 871 次
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee 等ICML 2023 · 被引用 764 次
- "Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AINithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong 等CHI 2021 · 被引用 725 次
相关 Paper
- Large Language Models in Qualitative Research: Uses, Tensions, and IntentionsHope Schroeder, Marianne Aubin Le Quéré, Casey Randazzo, David Mimno 等CHI 2025 · 被引用 40 次
- Interview-Informed Generative Agents for Product Discovery: A Validation StudyZichao Wang, Alexa F. SiuCHI 2026 · 被引用 1 次
- Collecting Qualitative Data at Scale with Large Language Models: A Case StudyAlejandro Cuevas Villalba, Jennifer V. Scurrell, Eva Maxfield Brown, Jason Entenmann 等CSCW 2025 · 被引用 14 次
- 'I'm Categorizing LLM as a Productivity Tool': Examining Ethics of LLM Use in HCI Research PracticesShivani Kapania, Ruiyi Wang, Toby Jia-Jun Li, Tianshi Li 等CSCW 2025 · 被引用 31 次
- Emerging Data Practices: Data Work in the Era of Large Language ModelsAdriana Alvarado Garcia, Heloisa Candello, Karla Badillo-Urquiola, Marisol Wong-VillacresCHI 2025 · 被引用 6 次
