Lune

ACL Findings2021

AgreeSum: Agreement-Oriented Multi-Document Summarization

Richard Yuanzhe Pang, Ádám Dániel Lelkes, Vinh Q. Tran, Cong Yu

Abstract

We aim to renew interest in a particular multidocument summarization (MDS) task which we call AgreeSum: agreement-oriented multidocument summarization. Given a cluster of articles, the goal is to provide abstractive summaries that represent information common and faithful to all input articles. Given the lack of existing datasets, we create a dataset for AgreeSum, and provide annotations on article-summary entailment relations for a subset of the clusters in the dataset. We aim to create strong baselines for the task by applying the top-performing pretrained singledocument summarization model PEGASUS onto AgreeSum, leveraging both annotated clusters by supervised losses, and unannotated clusters by T5-based entailment-related and language-related losses. Compared to other baselines, both automatic evaluation and human evaluation show better article-summary and cluster-summary entailment in generated summaries. On a separate note, we hope that our article-summary entailment annotations contribute to the community's effort in improving abstractive summarization faithfulness.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

7 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • A Unified Retrieval Framework with Document Ranking and EDU Filtering for Multi-document Summarization cites PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
  • Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback cites Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
  • Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback cites PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
  • Asking and Answering Questions to Evaluate the Factual Consistency of Summaries cites BERTScore: Evaluating Text Generation with BERT
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers7

  1. ExT5: Towards Extreme Multi-Task Scaling for Transfer LearningVamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao et al.ICLR 2022 · 237 citations
  2. Factually Consistent Summarization via Reinforcement Learning with Textual Entailment FeedbackPaul Roit, Johan Ferret, Lior Shani, Roee Aharoni et al.ACL 2023 · 21 citations
  3. Unsupervised Large Language Model Alignment for Information Retrieval via Contrastive FeedbackQian Dong, Yiding Liu, Qingyao Ai, Zhijing Wu et al.SIGIR 2024 · 9 citations
  4. Reward Gaming in Conditional Text GenerationRichard Yuanzhe Pang, Vishakh Padmakumar, Thibault Sellam, Ankur P. Parikh et al.ACL 2023 · 2 citations
  5. A Unified Retrieval Framework with Document Ranking and EDU Filtering for Multi-document SummarizationShiyin Tan, Jaeeon Park, Dongyuan Li, Renhe Jiang et al.SIGIR 2025

Builds on14

  1. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  2. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive SummarizationJingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. LiuICML 2020 · 2,453 citations
  3. Asking and Answering Questions to Evaluate the Factual Consistency of SummariesAlex Wang, Kyunghyun Cho, Mike LewisACL 2020 · 317 citations
  4. Heterogeneous Graph Neural Networks for Extractive Document SummarizationDanqing Wang, Pengfei Liu, Yining Zheng, Xipeng Qiu et al.ACL 2020 · 275 citations
  5. Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut et al.ACL 2020 · 168 citations

Related papers

  1. CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive SummarizationShuyang Cao, Lu WangEMNLP 2021 · 130 citations
  2. FaMeSumm: Investigating and Improving Faithfulness of Medical SummarizationNan Zhang, Yusen Zhang, Wu Guo, Prasenjit Mitra et al.EMNLP 2023 · 8 citations
  3. Towards Improving Faithfulness in Abstractive SummarizationXiuying Chen, Mingzhe Li, Xin Gao, Xiangliang ZhangNeurIPS 2022 · 39 citations
  4. ARMAN: Pre-training with Semantically Selecting and Reordering of Sentences for Persian Abstractive SummarizationAlireza Salemi, Emad Kebriaei, Ghazal Neisi Minaei, Azadeh ShakeryEMNLP 2021 · 4 citations
  5. Quiz-Style Question Generation for News StoriesÁdám D. Lelkes, Vinh Q. Tran, Cong YuWWW 2021 · 48 citations
Dusk over the sea between two cliffs drawn in fine vertical lines