ACL Findings2024
Do Androids Know They're Only Dreaming of Electric Sheep?
Sky CH-Wang, Benjamin Van Durme, Jason Eisner, Chris Kedzie
Abstract
We design probes trained on the internal representations of a transformer language model to predict its hallucinatory behavior on three grounded generation tasks. To train the probes, we annotate for span-level hallucination on both sampled (organic) and manually edited (synthetic) reference outputs. Our probes are narrowly trained and we find that they are sensitive to their training domain: they generalize poorly from one task to another or from synthetic to organic hallucinations. However, on in-domain data, they can reliably detect hallucinations at many transformer layers, achieving 95% of their peak performance as early as layer 4. Here, probing proves accurate for evaluating hallucination, outperforming several contemporary baselines and even surpassing an expert human annotator in response-level detection F1. Similarly, on span-level labeling, probes are on par or better than the expert annotator on two out of three generation tasks. Overall, we find that probing is a feasible and efficient alternative to language model hallucination evaluation when model states are available.
Code and data available at https://github.com/ microsoft/llm_generation_probes.
1 That is, do they know their electric sheep are ungrounded?
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
14 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation cites Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
- A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs cites FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
- A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs cites Uncertainty Estimation in Autoregressive Structured Prediction
- ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMs cites Uncertainty Estimation in Autoregressive Structured Prediction
Cited by top-tier papers14
- Attributive Reasoning for Hallucination Diagnosis of Large Language ModelsYuyan Chen, Zehao Li, Shuangjie You, Zhengyu Chen et al.AAAI 2025 · 33 citations
- ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMsZhenliang Zhang, Xinyu Hu, Huixuan Zhang, Junzhe Zhang et al.ACL 2025 · 17 citations
- Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention HeadsArtem Vazhentsev, Lyudmila Rvanova, Gleb Kuzmin, Ekaterina Fadeeva et al.ICML 2026 · 16 citations
- Prompt-Guided Internal States for Hallucination Detection of Large Language ModelsFujie Zhang, Peiqi Yu, Biao Yi, Baolei Zhang et al.ACL 2025 · 8 citations
- A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM OutputsArtem Shelmanov, Ekaterina Fadeeva, Akim Tsvigun, Ivan Tsvigun et al.EMNLP 2025 · 3 citations
Builds on22
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Uncertainty Estimation in Autoregressive Structured PredictionAndrey Malinin, Mark J. F. GalesICLR 2021 · 439 citations
- Asking and Answering Questions to Evaluate the Factual Consistency of SummariesAlex Wang, Kyunghyun Cho, Mike LewisACL 2020 · 317 citations
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text GenerationSewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis et al.EMNLP 2023 · 225 citations
Related papers
- Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTGuy Bar-Shalom, Fabrizio Frasca, Yaniv Galron, Yftah Ziser et al.NeurIPS 2025 · 17 citations
- On Early Detection of Hallucinations in Factual Question AnsweringBen Snyder, Marius Moisescu, Muhammad Bilal ZafarKDD 2024 · 10 citations
- Language Models Can Predict Their Own BehaviorDhananjay Ashok, Jonathan MayNeurIPS 2025 · 10 citations
- PROBE: PROcess-Based BEnchmark for Hallucination DetectionYu Zhang, Peter Belcak, Shizhe Diao, Yonggan Fu et al.ACL Findings 2026
- Quantifying Large Language Model Attacks Through the Lens of Model CognitionXiuming Liu, Chaoxiang He, Xuanran Yu, Jichen Chai et al.USENIX Security 2026
