ACL Findings2024
A Tale of Two Revisions: Summarizing Changes Across Document Versions
T. Y. S. S. Santosh, Natwar Modani, Apoorv Saxena
Abstract
Document revision is a crucial aspect of the writing process, particularly in collaborative environments where multiple authors contribute simultaneously. However, current tools lack an efficient way to provide a comprehensive overview of changes between versions, leading to difficulties in understanding revisions. To address this, we propose a novel task of providing thematic summary of changes between document versions, organizing individual edits based on shared themes. We assess capabilities of LLMs on this task and further introduce three strategies to tackle this task: (i) representing the input of two documents along with edits in the 'diff' format (ii) a two-stage task decomposition with individual edit description generation as an intermediate task and (iii) clustering based chunking and subsequent merging techniques for handling longer documents. Our experiments demonstrate the effectiveness of our approach in improving the model's capacity to handle this complex task. Additionally, we introduce ChangeSumm, a curated dataset comprising human-written thematic summaries for pairs of document versions, to facilitate evaluation and further research in this direction.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on11
- Document-Level Machine Translation with Large Language ModelsLongyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang et al.EMNLP 2023 · 129 citations
- LongBench: A Bilingual, Multitask Benchmark for Long Context UnderstandingYushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu et al.ACL 2024 · 94 citations
- BOOKSUM: A Collection of Datasets for Long-form Narrative SummarizationWojciech Kryscinski, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong et al.EMNLP Findings 2022 · 89 citations
- ContractNLI: A Dataset for Document-level Natural Language Inference for ContractsYuta Koreeda, Christopher D. ManningEMNLP Findings 2021 · 80 citations
- SCROLLS: Standardized CompaRison Over Long Language SequencesUri Shaham, Elad Segal, Maor Ivgi, Avia Efrat et al.EMNLP 2022 · 37 citations
Related papers
- Re3: A Holistic Framework and Dataset for Modeling Collaborative Document RevisionQian Ruan, Ilia Kuznetsov, Iryna GurevychACL 2024
- Automatic Document Sketching: Generating Drafts from Analogous TextsZeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang et al.ACL Findings 2021 · 3 citations
- Making Revisions Understandable: A Survey of Edit Intentions, Methods, and ApplicationsFangping Lan, Qi Zhang, Eduard C. DragutACL Findings 2026
- Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document RevisionsQian Ruan, Ilia Kuznetsov, Iryna GurevychEMNLP 2024 · 2 citations
- A System for Interleaving Discussion and Summarization in Online CollaborationSunny Tian, Amy X. Zhang, David R. KargerCSCW 2020 · 15 citations
