ACL Findings2025
Context-Aware Hierarchical Merging for Long Document Summarization
Litu Ou, Mirella Lapata
Abstract
Hierarchical Merging is a technique commonly used to summarize very long texts (>100K tokens) by breaking down the input into smaller sections, summarizing those sections individually, and then merging or combining those summaries into a final coherent summary. Although it helps address the limitations of large language models (LLMs) with fixed input length constraints, the recursive merging process can amplify LLM hallucinations, increasing the risk of factual inaccuracies. In this paper, we seek to mitigate hallucinations by enriching hierarchical merging with context from the source document. Specifically, we propose different approaches to contextual augmentation ranging from replacing intermediate summaries with relevant input context, to refining them while using the context as supporting evidence, and aligning them implicitly (via citations) to the input. Experimental results on datasets representing legal and narrative domains show that contextual augmentation consistently outperforms zero-shot and hierarchical merging baselines for the Llama 3.1 model family. Our analysis further reveals that refinement methods tend to perform best when paired with extractive summarization for identifying relevant input.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on21
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
- Transformers are RNNs: Fast Autoregressive Transformers with Linear AttentionAngelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, François FleuretICML 2020 · 2,665 citations
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
- YaRN: Efficient Context Window Extension of Large Language ModelsBowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico ShippoleICLR 2024 · 508 citations
Related papers
- Enhancing LLM's Cognition via StructurizationKai Liu, Zhihang Fu, Chao Chen, Wei Zhang et al.NeurIPS 2024 · 3 citations
- On Context Utilization in Summarization with Large Language ModelsMathieu Ravaut, Aixin Sun, Nancy F. Chen, Shafiq JotyACL 2024
- LLMInertia: Adaptive Counter-Inertial Reasoning to Improve Evidence Faithfulness in Large Language ModelsXinxin You, Xien Liu, Chenwei Yan, Siqi Song et al.ICML 2026
- DelucionQA: Detecting Hallucinations in Domain-specific Question AnsweringMobashir Sadat, Zhengyu Zhou, Lukas Lange, Jun Araki et al.EMNLP Findings 2023 · 27 citations
- Evaluation of LLMs in Medical Text Summarization: The Role of Vocabulary Adaptation in High OOV SettingsGunjan Balde, Soumyadeep Roy, Mainack Mondal, Niloy GangulyACL Findings 2025 · 5 citations
