Lune

EMNLP Findings2025

Understanding and Improving Information Preservation in Prompt Compression for LLMs

Weronika Lajewska, Momchil Hardalov, Laura Aina, Neha Anna John, Hang Su, Lluís Màrquez

Abstract

Recent advancements in large language models (LLMs) have enabled their successful application to a broad range of tasks. However, in information-intensive tasks, the prompt length can grow fast, leading to increased computational requirements, performance degradation, and induced biases from irrelevant or redundant information. Recently, various prompt compression techniques have been introduced to optimize the trade-off between reducing input length and retaining performance. We propose a holistic evaluation framework that allows for in-depth analysis of prompt compression methods. We focus on three key aspects, besides compression ratio: (i) downstream task performance, (ii) grounding in the input context, and (iii) information preservation. Using our framework, we analyze state-of-the-art soft and hard compression methods and show that some fail to preserve key details from the original prompt, limiting performance on complex tasks. By identifying these limitations, we are able to improve one soft prompting method by controlling compression granularity, achieving up to +23% in downstream performance, +8 BERTScore points in grounding, and 2.7× more entities preserved in compression. Ultimately, we find that the best effectiveness/compression rate trade-off is achieved with soft prompting combined with sequence-level training.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Large Language Models Can Be Easily Distracted by Irrelevant Context cites Language Models are Few-Shot Learners
  • Large Language Models Can Be Easily Distracted by Irrelevant Context cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Learning to Compress Prompts with Gist Tokens cites Language Models are Few-Shot Learners
  • Learning to Compress Prompts with Gist Tokens cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical StudyYizheng Sun, Hao Li, Chang Xu, Hongpeng Zhou et al.EMNLP 2025

Builds on24

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  4. Large Language Models Can Be Easily Distracted by Irrelevant ContextFreda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales et al.ICML 2023 · 970 citations
  5. Learning to Compress Prompts with Gist TokensJesse Mu, Xiang Li, Noah D. GoodmanNeurIPS 2023 · 488 citations

Related papers

  1. From Reading to Compressing: Exploring the Multi-document Reader for Prompt CompressionEunseong Choi, Sunkyung Lee, Minjin Choi, June Park et al.EMNLP Findings 2024
  2. GMSA: Enhancing Context Compression via Group Merging and Layer Semantic AlignmentJiwei Tang, Zhicheng Zhang, Shunlong Wu, Jingheng Ye et al.ACL 2026 · 24 citations
  3. LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt CompressionZhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia et al.ACL Findings 2024 · 58 citations
  4. Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM InferenceBarys Liskavets, Maxim Ushakov, Shuvendu Roy, Mark Klibanov et al.AAAI 2025 · 41 citations
  5. No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt OptimizationWenhang Shi, Yiren Chen, Shuqing Bian, Xinyi Zhang et al.NeurIPS 2025 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines