PrimeX: A Dataset of Worldview, Opinion, and Explanation
Rik Koncel-Kedziorski, Brihi Joshi, Tim Paek
摘要
As the adoption of language models advances, so does the need to better represent individual users to the model. Are there aspects of an individual's belief system that a language model can utilize for improved alignment? Following prior research, we investigate this question in the domain of opinion prediction by developing PRIMEX, a dataset of public opinion survey data from 858 US residents with two additional sources of belief information: written explanations from the respondents for why they hold specific opinions, and the Primal World Belief survey for assessing respondent worldview. We provide an extensive initial analysis of our data and show the value of belief explanations and worldview for personalizing language models. Our results demonstrate how the additional belief information in PRIMEX can benefit both the NLP and psychological research communities, opening up avenues for further study. Public Opinion Surveys on multiple topics 858 Participants of various age, gender, education, race, & US region 1 2 3 Free-response explanations of 3 questions per survey Good vs. Bad Safe vs. Dangerous Enticing vs. Dull Alive vs. Mechanistic
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
它引用的 Paper13
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 24,707 次引用
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 22,562 次引用
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 10,924 次引用
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee 等ICML 2023 · 764 次引用
- Fine-Grained Human Feedback Gives Better Rewards for Language Model TrainingZeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri 等NeurIPS 2023 · 516 次引用
相关 Paper
- Aligning Language Models to User OpinionsEunJeong Hwang, Bodhisattwa Prasad Majumder, Niket TandonEMNLP Findings 2023 · 22 次引用
- Improving Language Model Personas via Rationalization with Psychological ScaffoldsBrihi Joshi, Xiang Ren, Swabha Swayamdipta, Rik Koncel-Kedziorski 等EMNLP Findings 2025
- Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief NetworksYun-Shiuan Chuang, Krirk Nirunwiroj, Zach Studdiford, Agam Goyal 等EMNLP Findings 2024 · 8 次引用
- Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMsTristan Williams, Franziska Weeber, Sebastian Padó, Alan AkbikACL Findings 2026
- BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of BeliefNora Kassner, Oyvind Tafjord, Hinrich Schütze, Peter ClarkEMNLP 2021 · 2 次引用
