Whom to Query for What: Adaptive Group Elicitation via Multi-Turn LLM Interactions
Ruomeng Ding, Tianwei Gao, Tom Zollo, Eitan Bachmat, Richard Zemel, Xinyu Yang
Abstract
Eliciting information to reduce uncertainty about latent group-level properties is a central problem in collective assessment, preference modeling, and opinion aggregation, and is especially important in survey-based studies. While natural language interactions provide a flexible interface, existing methods typically rely on fixed questionnaires and static respondent sets, and do not adapt to partial or missing responses across rounds. To address this gap, we study adaptive information elicitation through multi-turn interactions between a large language model and a group of individuals, where both queries and respondents are adaptively selected to infer latent group properties. We propose a theoretically grounded framework that, at each round, jointly selects a query and a subset of respondents based on previously observed responses to efficiently reduce uncertainty about a target latent quantity (e.g., group-level political inclination). Motivated by practical survey constraints, such as limited questions and costly participation, our strategy maximizes information gain under a fixed budget. To handle missing and incomplete responses, we combine graph neural networks for aggregating/imputing partial group information with an information-theoretic criterion that guides per-round selection. Across three real-world opinion datasets, we achieve consistent improvements in population-level response prediction under constrained budgets, including over a 12% relative gain on CES at a 10% respondent budget.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 2b3e209f-d191-468b-b451-da1275b7e969Builds on7
- An Explanation of In-context Learning as Implicit Bayesian InferenceSang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu MaICLR 2022 · 1,030 citations
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
- Debating with More Persuasive LLMs Leads to More Truthful AnswersAkbir Khan, John Hughes, Dan Valentine, Laura Ruis et al.ICML 2024 · 244 citations
- Deep Adaptive Design: Amortizing Sequential Bayesian Experimental DesignAdam Foster, Desi R. Ivanova, Ilyas Malik, Tom RainforthICML 2021 · 119 citations
- Implicit Deep Adaptive Design: Policy-Based Experimental Design without LikelihoodsDesi R. Ivanova, Adam Foster, Steven Kleinegesse, Michael U. Gutmann et al.NeurIPS 2021 · 81 citations
Related papers
- Adaptive Elicitation of Latent Information Using Natural LanguageJimmy Wang, Thomas P. Zollo, Richard S. Zemel, Hongseok NamkoongICML 2025
- Adaptive Querying with AI Persona PriorsKaizheng Wang, Yuhang Wu, Assaf ZeeviICML 2026
- Efficient Exploration for LLMsVikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao, Benjamin Van RoyICML 2024 · 45 citations
- BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental DesignDeepro Choudhury, Sinead Williamson, Adam Golinski, Ning Miao et al.ICLR 2026 · 24 citations
- Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and RectificationStefan Krsteski, Giuseppe Russo, Serina Chang, Robert West et al.ACL 2026 · 10 citations
