Lune

EMNLP2025Top-tier venue

PrimeX: A Dataset of Worldview, Opinion, and Explanation

Rik Koncel-Kedziorski, Brihi Joshi, Tim Paek

Abstract

As the adoption of language models advances, so does the need to better represent individual users to the model. Are there aspects of an individual's belief system that a language model can utilize for improved alignment? Following prior research, we investigate this question in the domain of opinion prediction by developing PRIMEX, a dataset of public opinion survey data from 858 US residents with two additional sources of belief information: written explanations from the respondents for why they hold specific opinions, and the Primal World Belief survey for assessing respondent worldview. We provide an extensive initial analysis of our data and show the value of belief explanations and worldview for personalizing language models. Our results demonstrate how the additional belief information in PRIMEX can benefit both the NLP and psychological research communities, opening up avenues for further study. Public Opinion Surveys on multiple topics 858 Participants of various age, gender, education, race, & US region 1 2 3 Free-response explanations of 3 questions per survey Good vs. Bad Safe vs. Dangerous Enticing vs. Dull Alive vs. Mechanistic

Ask Lune

Ask anything. The answer quotes the paper.

Lune reads all of it and the papers around it, then shows you the passages behind every answer.

Questions to start from

Lune answers from

the full paper · related work

Get the answer

Free to start. No credit card required.

Builds on13

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  4. Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
  5. Fine-Grained Human Feedback Gives Better Rewards for Language Model TrainingZeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri et al.NeurIPS 2023 · 516 citations

Related papers

  1. Aligning Language Models to User OpinionsEunJeong Hwang, Bodhisattwa Prasad Majumder, Niket TandonEMNLP Findings 2023 · 22 citations
  2. Improving Language Model Personas via Rationalization with Psychological ScaffoldsBrihi Joshi, Xiang Ren, Swabha Swayamdipta, Rik Koncel-Kedziorski et al.EMNLP Findings 2025
  3. Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief NetworksYun-Shiuan Chuang, Krirk Nirunwiroj, Zach Studdiford, Agam Goyal et al.EMNLP Findings 2024 · 8 citations
  4. Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMsTristan Williams, Franziska Weeber, Sebastian Padó, Alan AkbikACL Findings 2026
  5. BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of BeliefNora Kassner, Oyvind Tafjord, Hinrich Schütze, Peter ClarkEMNLP 2021 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines