ACL Findings2026
Enhancing Factuality through Consensus and Consistency in Summarization Using Minimum Bayes Risk Decoding
Riza Setiawan Soetedjo, Yusuke Sakai, Hidetaka Kamigaito, Jingun Kwon, Manabu Okumura, Taro Watanabe
Abstract
Improving the quality of model-generated summaries, especially factuality, the accuracy of a summary with respect to its source content, remains a challenge. While reranking could select the optimal output from multiple generated candidates, it is limited to only using the source as guidance, resulting in unreliable summaries. To address this limitation, we propose Con-SUM that reranks candidate summaries by considering two factors: consistency to the source document and consensus among the other candidates. Consensus is established using Minimum Bayes Risk (MBR) decoding over the set of generated summaries, while ensuring consistency by employing factuality-aware metrics that compare the summary against the source. Rigorous testing demonstrates that our system is competitive with existing methods, with human evaluations further confirming that its generated summaries are preferred over those from other systems. Our code is available at https://github.com/naist-nlp/ConSUM.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on9
- The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
- Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- On Faithfulness and Factuality in Abstractive SummarizationJoshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan T. McDonaldACL 2020 · 54 citations
- Truncation Sampling as Language Model DesmoothingJohn Hewitt, Christopher D. Manning, Percy LiangEMNLP Findings 2022 · 26 citations
Related papers
- X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive SummarizationSubhajit Chaudhury, Sarathkrishna Swaminathan, R. Chulaka Gunasekara, Maxwell Crouse et al.EMNLP 2022 · 13 citations
- Improving Factual Consistency of Abstractive Summarization via Question AnsweringFeng Nan, Cícero Nogueira dos Santos, Henghui Zhu, Patrick Ng et al.ACL 2021
- Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel ModelNico Daheim, David Thulke, Christian Dugast, Hermann NeyEMNLP Findings 2022 · 1 citation
- Evaluating the Factual Consistency of Abstractive Text SummarizationWojciech Kryscinski, Bryan McCann, Caiming Xiong, Richard SocherEMNLP 2020 · 67 citations
- Factually Consistent Summarization via Reinforcement Learning with Textual Entailment FeedbackPaul Roit, Johan Ferret, Lior Shani, Roee Aharoni et al.ACL 2023 · 21 citations
