Lune

ACL Findings2025

Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks

Yuntai Bao, Xuhong Zhang, Tianyu Du, Xinkui Zhao, Zhengwen Feng, Hao Peng, Jianwei Yin

Abstract

Large language models (LLMs) are trained on extensive datasets that encapsulate substantial world knowledge. However, their outputs often include confidently stated inaccuracies. Earlier works suggest that LLMs encode truthfulness as a distinct linear feature, termed the "truth direction", which can classify truthfulness reliably. We address several open questions about the truth direction: (i) whether LLMs universally exhibit consistent truth directions; (ii) whether sophisticated probing techniques are necessary to identify truth directions; and (iii) how the truth direction generalizes across diverse contexts. Our findings reveal that not all LLMs exhibit consistent truth directions, with stronger representations observed in more capable models, particularly in the context of logical negation. Additionally, we demonstrate that truthfulness probes trained on declarative atomic statements can generalize effectively to logical transformations, question-answering tasks, in-context learning, and external knowledge sources. Finally, we explore the practical application of truthfulness probes in selective question-answering, illustrating their potential to improve user trust in LLM outputs. These results advance our understanding of truth directions and provide new insights into the internal representations of LLM beliefs. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
  • Inference-Time Intervention: Eliciting Truthful Answers from a Language Model cites Measuring Massive Multitask Language Understanding
  • Inference-Time Intervention: Eliciting Truthful Answers from a Language Model cites TruthfulQA: Measuring How Models Mimic Human Falsehoods
  • Truth is Universal: Robust Detection of Lies in LLMs cites The Internal State of an LLM Knows When It's Lying
  • Truth is Universal: Robust Detection of Lies in LLMs cites Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
  • How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs cites The Internal State of an LLM Knows When It's Lying
  • How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs cites Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
  • How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs cites Truth is Universal: Robust Detection of Lies in LLMs
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Truth as a Trajectory: What Internal Representations Reveal About Large Language Model ReasoningHamed Damirchi, Ignacio Meza De La Jara, Ehsan Abbasnejad, Afshar Shamsi et al.ACL 2026 · 8 citations
  2. How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMsShivam Adarsh, Maria Maistro, Christina LiomaACL 2026

Builds on15

  1. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  2. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  3. Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
  4. The Internal State of an LLM Knows When It's LyingAmos Azaria, Tom M. MitchellEMNLP Findings 2023 · 177 citations
  5. Truth is Universal: Robust Detection of Lies in LLMsLennart Bürger, Fred A. Hamprecht, Boaz NadlerNeurIPS 2024 · 93 citations

Related papers

  1. LLM Beliefs Are in Their HeadsAlessandro Corona Mendozza, Anders SøgaardACL 2026
  2. Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMsGiovanni Servedio, Alessandro De Bellis, Dario Di Palma, Vito Walter Anelli et al.ACL 2025 · 9 citations
  3. HyperEdit: Mitigating Hallucinations of Large Language Models via Hyperbolic Representation EditingTongxu Lin, Junping Du, Zhe Xue, Meiyu Liang et al.KDD 2026
  4. On the Universal Truthfulness Hyperplane Inside LLMsJunteng Liu, Shiqi Chen, Yu Cheng, Junxian HeEMNLP 2024
  5. Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty ExpressionFarima Fatahi Bayat, Xin Liu, H. V. Jagadish, Lu WangACL Findings 2024 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines