Lune

EMNLP Findings2025

SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMs

Samir Abdaljalil, Filippo Pallucchini, Andrea Seveso, Hasan Kurban, Fabio Mercorio, Erchin Serpedin

Abstract

Despite their state-of-the-art capabilities, Large Language Models (LLMs) often suffer from hallucinations, which can compromise their reliability in critical applications. In this work, we propose SAFE, a novel framework for detecting and mitigating hallucinations by leveraging Sparse Autoencoders (SAEs). While hallucination detection techniques and SAEs have been explored independently, their synergistic application in a comprehensive system, particularly for hallucination-aware query enrichment, has not been fully investigated. To validate the effectiveness of SAFE, we evaluate it on two models with available SAEs across four diverse cross-domain datasets designed to assess hallucination problems. Empirical results demonstrate that SAFE consistently improves query generation accuracy and mitigates hallucinations across all datasets, achieving accuracy improvements of up to 29.45%.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders cites Sparse Autoencoders Find Highly Interpretable Features in Language Models
  • Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Self-Refine: Iterative Refinement with Self-Feedback cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models cites Sparse Autoencoders Find Highly Interpretable Features in Language Models
  • Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Interpretability Transfer from Language to Vision via Sparse Autoencoders cites Sparse Autoencoders Find Highly Interpretable Features in Language Models
  • LIMA: Less Is More for Alignment cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Auditing Meta-Cognitive Hallucinations in Reasoning Large Language ModelsHaolang Lu, Yilian Liu, Jingxin Xu, Guoshun Nan et al.NeurIPS 2025 · 23 citations
  2. Toward Faithful Retrieval-Augmented Generation with Sparse AutoencodersGuangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha et al.ICLR 2026 · 8 citations
  3. Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse AutoencodersXiangchen Song, Aashiq Muhamed, Yujia Zheng, Lingjing Kong et al.ACL 2026
  4. Interpretability Transfer from Language to Vision via Sparse AutoencodersAlexey Kravets, Da Li, Chuan Li, Da Chen et al.ICML 2026

Builds on17

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  4. LIMA: Less Is More for AlignmentChunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer et al.NeurIPS 2023 · 1,486 citations
  5. Sparse Autoencoders Find Highly Interpretable Features in Language ModelsRobert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart et al.ICLR 2024 · 1,072 citations

Related papers

  1. Steering LVLMs via Sparse Autoencoder for Hallucination MitigationZhenglin Hua, Jinghan He, Zijun Yao, Tianxu Han et al.EMNLP Findings 2025
  2. The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language ModelsJunyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng et al.ACL 2024 · 49 citations
  3. PHPFND: Detecting Fake News via Post-Hoc Processing of LLMs HallucinationJinke Ma, Jiachen Ma, Wei Zhang, Yong LiuAAAI 2026
  4. HalluClean: A Unified Framework to Combat Hallucinations in LLMsYaxin Zhao, Yu ZhangAAAI 2026
  5. ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language ModelsYuzhe Gu, Ziwei Ji, Wenwei Zhang, Chengqi Lyu et al.NeurIPS 2024 · 20 citations
Dusk over the sea between two cliffs drawn in fine vertical lines