EMNLP Findings2023
Improving Consistency for Text Summarization with Energy Functions
Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang
Abstract
Current abstractive summarization models often generate inconsistent content, i.e. texts that are not directly inferable from the source document, are not consistent with respect to world knowledge, or are self-contradictory. These inconsistencies motivate a new consistency taxonomy that we define as faithfulness, factuality, and self-supportiveness. However, most recent work on reducing inconsistency in document summarization only focuses on faithfulness detection and correction while ignoring other inconsistency phenomena, which limits the model's scalability. To improve the general consistency we introduce EnergySum, where we apply the Residual Energy-based Model by designing energy scorers that reflect each type of consistency. These energy scores are utilized in candidate re-ranking during the sampling process. Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics cites BERTScore: Evaluating Text Generation with BERT
- COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics cites Residual Energy-Based Models for Text Generation
Builds on16
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- COLD Decoding: Energy-based Constrained Text Generation with Langevin DynamicsLianhui Qin, Sean Welleck, Daniel Khashabi, Yejin ChoiNeurIPS 2022 · 217 citations
- Residual Energy-Based Models for Text GenerationYuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam et al.ICLR 2020 · 147 citations
Related papers
- X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive SummarizationSubhajit Chaudhury, Sarathkrishna Swaminathan, R. Chulaka Gunasekara, Maxwell Crouse et al.EMNLP 2022 · 13 citations
- Improving Factual Consistency of Abstractive Summarization via Question AnsweringFeng Nan, Cícero Nogueira dos Santos, Henghui Zhu, Patrick Ng et al.ACL 2021
- Multi-Fact Correction in Abstractive Text SummarizationYue Dong, Shuohang Wang, Zhe Gan, Yu Cheng et al.EMNLP 2020 · 99 citations
- Factual Relation Discrimination for Factuality-oriented Abstractive SummarizationZhiguang Gao, Peifeng Li, Feng Jiang, Xiaomin Chu et al.EMNLP Findings 2023 · 1 citation
- Fine-grained Factual Consistency Assessment for Abstractive Summarization ModelsSen Zhang, Jianwei Niu, Chuyuan WeiEMNLP 2021 · 8 citations
