Lune

EMNLP Findings2025

Evaluating Large Language Models for Belief Inference: Mapping Belief Networks at Scale

Trisevgeni Papakonstantinou, Antonina Zhiteneva, Ana Yutong Ma, Derek Powell, Zachary Horne

Abstract

Beliefs are interconnected, influencing how people process and update what they think. To study the interconnectedness of beliefs at scale, we introduce a novel analytical pipeline leveraging a finetuned GPT-4o model to infer belief structures from large-scale social media data. We evaluate the model's performance by (1) comparing it to human annotated data (2) and its inferences to human-generated survey data. Our results show that a fine-tuned GPT-4o model can effectively recover belief structures, allowing for a level of scalability and efficiency that is impossible using traditional survey methods of data collection. This work demonstrates the potential for large language models to perform belief inference tasks and provides a framework for future research on the analysis of belief structures.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Related papers

  1. Language Models Represent Beliefs of Self and OthersWentao Zhu, Zhining Zhang, Yizhou WangICML 2024 · 24 citations
  2. Co-Writing with Opinionated Language Models Affects Users' ViewsMaurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson et al.CHI 2023 · 249 citations
  3. Beyond Evidence: Belief-Chain Conditioning for Persuasive Misinformation Debunking ExplanationYi-Li Hsu, Li-Wun Chang, Chun-Yu Hsu, Jane Wei-Kuan Shih et al.ACL Findings 2026
  4. Brittle Minds, Fixable Activations: Understanding Belief Representations in Language ModelsMatteo Bortoletto, Constantin Ruhdorfer, Lei Shi, Andreas BullingEMNLP Findings 2025
  5. CONSTRUCTURE: Benchmarking CONcept STRUCTUre REasoning for Multimodal Large Language ModelsZhiwei Zha, Xiangru Zhu, Yuanyi Xu, Chenghua Huang et al.EMNLP Findings 2024
Dusk over the sea between two cliffs drawn in fine vertical lines