ACL Findings2024
Better Late Than Never: Model-Agnostic Hallucination Post-Processing Framework Towards Clinical Text Summarization
Songda Li, Yunqi Zhang, Chunyuan Deng, Yake Niu, Hui Zhao
Abstract
Clinical text summarization has proven successful in generating concise and coherent summaries. However, these summaries may include unintended text with hallucinations, which can mislead clinicians and patients. Existing methods for mitigating hallucinations can be categorized into task-specific and task-agnostic approaches. Task-specific methods lack versatility for real-world applicability. Meanwhile, task-agnostic methods are not model-agnostic, so they require retraining for different models, resulting in considerable computational costs. To address these challenges, we propose MEDAL, a model-agnostic framework designed to post-process medical hallucinations. MEDAL can seamlessly integrate with any medical summarization model, requiring no additional computational overhead. MEDAL comprises a medical infilling model and a hallucination correction model. The infilling model generates non-factual summaries with common errors to train the correction model. The correction model is incorporated with a selfexamination mechanism to activate its cognitive capability. We conduct comprehensive experiments using 11 widely accepted metrics on 7 baseline models across 3 medical text summarization tasks. MEDAL demonstrates superior performance in correcting hallucinations when applied to summaries generated by pre-trained language models and large language models. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- BARTScore: Evaluating Generated Text as Text Generation cites BERTScore: Evaluating Text Generation with BERT
- BARTScore: Evaluating Generated Text as Text Generation cites PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
- Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports cites BERTScore: Evaluating Text Generation with BERT
- How Private are Language Models in Abstractive Summarization? cites BERTScore: Evaluating Text Generation with BERT
- How Private are Language Models in Abstractive Summarization? cites PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
Cited by top-tier papers1
Builds on19
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive SummarizationJingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. LiuICML 2020 · 2,453 citations
- BARTScore: Evaluating Generated Text as Text GenerationWeizhe Yuan, Graham Neubig, Pengfei LiuNeurIPS 2021 · 1,143 citations
- MedDialog: Large-scale Medical Dialogue DatasetsGuangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang et al.EMNLP 2020 · 163 citations
- Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology ReportsYuhao Zhang, Derek Merck, Emily Bao Tsai, Christopher D. Manning et al.ACL 2020 · 160 citations
Related papers
- Towards Mitigating LLM Hallucination via Self ReflectionZiwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee et al.EMNLP Findings 2023 · 274 citations
- Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive SummarizationMeng Cao, Yue Dong, Jackie Chi Kit CheungACL 2022
- Hallucinations in LLM-Based Code Summarization: Unveiling, Detection, and MitigationGuanghua Wan, Yuanning Feng, Yao Wan, Zhaoyang Chu et al.FSE 2026
- ETF: An Entity Tracing Framework for Hallucination Detection in Code SummariesKishan Maharaj, Vitobha Munigala, Srikanth G. Tamilselvam, Prince Kumar et al.ACL 2025 · 4 citations
- Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model InfillingVidhisha Balachandran, Hannaneh Hajishirzi, William W. Cohen, Yulia TsvetkovEMNLP 2022 · 27 citations
