PICTURE: Enhancing Theory-of-Mind in Large Language Models by Revealing, Not Hiding, Characters' Lack of Knowledge
Eojin Jeon, SangKeun Lee
Abstract
Simulating human-like Theory of Mind (ToM) has been a longstanding problem in natural language processing (NLP). To address this, existing works introduce a reasoning step of event hiding (a.k.a. perspective-taking), where events unknown to a character are removed before question answering. However, resorting to event hiding for ToM reasoning presents a performance degradation issue due to the strict output format constraints involved in event hiding. To mitigate this issue, we propose generating perspective-taking outputs as free-form explanations without event hiding, but this poses a notable yet underexplored challenge: LLMs need to inhibit responses to events unknown to characters, because the absence of event hiding exposes LLMs to these events throughout reasoning. To address this challenge, we hypothesize and empirically verify that LLMs can achieve such inhibition if a character’s lack of knowledge about events is made explicit during reasoning. Based on this finding, we introduce P ICTURE , a new prompting method that enables LLMs to generate a character’s lack of knowledge within free-form CoT. Experimental results show that P ICTURE outperforms existing prompting methods by an average of 7.3% on false-belief tasks. 1
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on13
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language ModelsLei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu et al.ACL 2023 · 249 citations
- Theory of Mind for Multi-Agent Collaboration via Large Language ModelsHuao Li, Yu Quan Chong, Simon Stepputtis, Joseph Campbell et al.EMNLP 2023 · 57 citations
- Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and FutureZheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu et al.ACL 2024 · 36 citations
Related papers
- Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind CapabilitiesAlex Wilf, Sihyun Shawn Lee, Paul Pu Liang, Louis-Philippe MorencyACL 2024
- Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language ModelsChani Jung, Dongkwan Kim, Jiho Jin, Jiseon Kim et al.EMNLP 2024 · 2 citations
- Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief TrackerMelanie Sclar, Sachin Kumar, Peter West, Alane Suhr et al.ACL 2023 · 21 citations
- Making Large Language Models Better Reasoners with Orchestrated Streaming ExperiencesXiangyang Liu, Junliang He, Xipeng QiuEMNLP 2024
- Video-Only ToM: Enhancing Theory of Mind in Multimodal Large Language ModelsSiqi Liu, Xinyang Li, Bochao Zou, Junbao Zhuo et al.CVPR 2026
