Lune

ACL Findings2025

Explainable Hallucination through Natural Language Inference Mapping

Wei-Fan Chen, Zhixue Zhao, Akbar Karimi, Lucie Flek

Abstract

Large language models (LLMs) often generate hallucinated content, making it crucial to identify and quantify inconsistencies in their outputs. We introduce HaluMap, a post-hoc framework that detects hallucinations by mapping entailment and contradiction relations between source inputs and generated outputs using a natural language inference (NLI) model. To improve reliability, we propose a calibration step leveraging intra-text relations to refine predictions. HaluMap outperforms stateof-the-art NLI-based methods by five percentage points compared to other training-free approaches, while providing clear, interpretable explanations. As a training-free and modelagnostic approach, HaluMap offers a practical solution for verifying LLM outputs across diverse NLP tasks. The resources of this paper are available at https://github.com/ caisa-lab/acl25-halumap.

Ask Lune

Ask anything. The answer quotes the paper.

Lune reads all of it and the papers around it, then shows you the passages behind every answer.

Questions to start from

Lune answers from

the full paper · related work

Get the answer

Free to start. No credit card required.

Builds on13

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
  3. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 citations
  4. Chain-of-Verification Reduces Hallucination in Large Language ModelsShehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu et al.ACL Findings 2024 · 281 citations
  5. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text GenerationSewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis et al.EMNLP 2023 · 225 citations

Related papers

  1. HalluClean: A Unified Framework to Combat Hallucinations in LLMsYaxin Zhao, Yu ZhangAAAI 2026
  2. HaDeMiF: Hallucination Detection and Mitigation in Large Language ModelsXiaoling Zhou, Mingjie Zhang, Zhemg Lee, Wei Ye et al.ICLR 2025
  3. Benchmarking Query-Conditioned Natural Language InferenceMarc E. Canby, Xinchi Chen, Xing Niu, Jifan Chen et al.ACL Findings 2025 · 1 citation
  4. The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language ModelsJunyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng et al.ACL 2024 · 49 citations
  5. Reference-free Hallucination Detection for Large Vision-Language ModelsQing Li, Jiahui Geng, Chenyang Lyu, Derui Zhu et al.EMNLP Findings 2024 · 16 citations
Dusk over the sea between two cliffs drawn in fine vertical lines