Lune

EMNLP Findings2021

Leveraging Information Bottleneck for Scientific Document Summarization

Jiaxin Ju, Ming Liu, Huan Yee Koh, Yuan Jin, Lan Du, Shirui Pan

Abstract

This paper presents an unsupervised extractive approach to summarize scientific long documents based on the Information Bottleneck principle. Inspired by previous work which uses the Information Bottleneck principle for sentence compression, we extend it to document level summarization with two separate steps. In the first step, we use signal(s) as queries to retrieve the key content from the source document. Then, a pre-trained language model conducts further sentence search and edit to return the final extracted summaries. Importantly, our work can be flexibly extended to a multi-view framework by different signals. Automatic evaluation on three scientific document datasets verifies the effectiveness of the proposed framework. The further human evaluation suggests that the extracted summaries cover more content aspects than previous systems.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • How Far are We from Robust Long Abstractive Summarization? cites Big Bird: Transformers for Longer Sequences
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language UnderstandingJunda Wu, Tong Yu, Rui Wang, Zhao Song et al.NeurIPS 2023 · 48 citations
  2. HEGEL: Hypergraph Transformer for Long Document SummarizationHaopeng Zhang, Xiao Liu, Jiawei ZhangEMNLP 2022 · 33 citations
  3. How Far are We from Robust Long Abstractive Summarization?Huan Yee Koh, Jiaxin Ju, He Zhang, Ming Liu et al.EMNLP 2022 · 16 citations
  4. Question-Answer Extraction from Scientific Articles Using Knowledge Graphs and Large Language ModelsHosein Azarbonyad, Zi Long Zhu, Georgios Cheirmpos, Zubair Afzal et al.SIGIR 2025 · 4 citations
  5. What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific PresentationsDongqi Liu, Chenxi Whitehouse, Xi Yu, Louis Mahon et al.ACL 2025

Builds on4

  1. Big Bird: Transformers for Longer SequencesManzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie et al.NeurIPS 2020 · 3,159 citations
  2. Learning Robust Representations via Multi-View Information BottleneckMarco Federici, Anjan Dutta, Patrick Forré, Nate Kushman et al.ICLR 2020 · 330 citations
  3. Intrinsic Evaluation of Summarization DatasetsRishi Bommasani, Claire CardieEMNLP 2020 · 52 citations
  4. Phase Transitions for the Information Bottleneck in Representation LearningTailin Wu, Ian S. FischerICLR 2020 · 31 citations

Related papers

  1. Scientific Paper Extractive Summarization Enhanced by Citation GraphsXiuying Chen, Mingzhe Li, Shen Gao, Rui Yan et al.EMNLP 2022 · 8 citations
  2. Explanation Regeneration via Information BottleneckQintong Li, Zhiyong Wu, Lingpeng Kong, Wei BiACL Findings 2023 · 4 citations
  3. MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase ExtractionLinhan Zhang, Qian Chen, Wen Wang, Chong Deng et al.ACL Findings 2022 · 63 citations
  4. SPECTER: Document-level Representation Learning using Citation-informed TransformersArman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey et al.ACL 2020 · 20 citations
  5. Explaining Relationships Between Scientific DocumentsKelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo et al.ACL 2021
Dusk over the sea between two cliffs drawn in fine vertical lines