Lune

EMNLP Findings2024

From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression

Eunseong Choi, Sunkyung Lee, Minjin Choi, June Park, Jongwuk Lee

Abstract

Large language models (LLMs) have achieved significant performance gains using advanced prompting techniques over various tasks. However, the increasing length of prompts leads to high computational costs and often obscures crucial information. Prompt compression has been proposed to alleviate these issues, but it faces challenges in (i) capturing the global context and (ii) training the compressor effectively. To tackle these challenges, we introduce a novel prompt compression method, namely Reading To Compressing (R2C), utilizing the Fusionin-Decoder (FiD) architecture to identify the important information in the prompt. Specifically, the cross-attention scores of the FiD are used to discern essential chunks and sentences from the prompt. R2C effectively captures the global context without compromising semantic consistency while detouring the necessity of pseudo-labels for training the compressor. Empirical results show that R2C retains key contexts, enhancing the LLM performance by 6% in out-of-domain evaluations while reducing the prompt length by 80%.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token cites Learning to Compress Prompts with Gist Tokens
  • Learning to Compress Prompts with Gist Tokens cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering cites xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
  • IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering cites Learning to Compress Prompts with Gist Tokens
  • IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering cites RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation
  • RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation cites Learning to Compress Prompts with Gist Tokens
  • Draft-based Approximate Inference for LLMs cites Unsupervised Cross-lingual Representation Learning at Scale
  • Conflict-Aware Soft Prompting for Retrieval-Augmented Generation cites xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
  • Conflict-Aware Soft Prompting for Retrieval-Augmented Generation cites Learning to Compress Prompts with Gist Tokens
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Draft-based Approximate Inference for LLMsKevin Galim, Ethan Ewer, Wonjun Kang, Minjae Lee et al.ICLR 2026 · 5 citations
  2. Conflict-Aware Soft Prompting for Retrieval-Augmented GenerationEunseong Choi, June Park, Hyeri Lee, Jongwuk LeeEMNLP 2025 · 1 citation
  3. IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question AnsweringJungMin Yun, YoungBin KimACL 2026

Builds on14

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
  3. Learning to Compress Prompts with Gist TokensJesse Mu, Xiang Li, Noah D. GoodmanNeurIPS 2023 · 488 citations
  4. RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective AugmentationFangyuan Xu, Weijia Shi, Eunsol ChoiICLR 2024 · 260 citations
  5. xRAG: Extreme Context Compression for Retrieval-augmented Generation with One TokenXin Cheng, Xun Wang, Xingxing Zhang, Tao Ge et al.NeurIPS 2024 · 156 citations

Related papers

  1. Understanding and Improving Information Preservation in Prompt Compression for LLMsWeronika Lajewska, Momchil Hardalov, Laura Aina, Neha Anna John et al.EMNLP Findings 2025 · 3 citations
  2. Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM InferenceBarys Liskavets, Maxim Ushakov, Shuvendu Roy, Mark Klibanov et al.AAAI 2025 · 41 citations
  3. Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer InferenceWeizhi Fei, Xueyan Niu, Guoqing Xie, Yingqing Liu et al.NeurIPS 2025 · 14 citations
  4. Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and TransferabilityTsz Ting Chung, Leyang Cui, Lemao Liu, Xinting Huang et al.EMNLP Findings 2024 · 1 citation
  5. Leveraging Attention to Effectively Compress Prompts for Long-Context LLMsYunlong Zhao, Haoran Wu, Bo XuAAAI 2025 · 10 citations
Dusk over the sea between two cliffs drawn in fine vertical lines