ACL Findings2025
TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning
Shivam Shandilya, Menglin Xia, Supriyo Ghosh, Huiqiang Jiang, Jue Zhang, Qianhui Wu, Victor Rühle, Saravan Rajmohan
Abstract
The increasing prevalence of large language models (LLMs) such as GPT-4 in various applications has led to a surge in the size of prompts required for optimal performance, leading to challenges in computational efficiency. Prompt compression aims to reduce the inference cost by minimizing input tokens without compromising on the task performance. However, existing prompt compression techniques either rely on sub-optimal metrics such as information entropy or model it as a task-agnostic token classification problem that fails to capture task-specific information.
To address these issues, we propose a novel and efficient reinforcement learning (RL) based task-aware prompt compression method. To ensure low latency requirements, we leverage existing Transformer encoder-based token classification model while guiding the learning process with task-specific reward signals using lightweight REINFORCE algorithm. We evaluate the performance of our method on three diverse and challenging tasks including text summarization, question answering and code summarization. We demonstrate that our RL-guided compression method improves the task performance by 8% -189% across these three scenarios over state-of-the-art compression techniques while satisfying the same compression rate and latency requirements.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation cites Language Models are Few-Shot Learners
- How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- What Makes a Good Natural Language Prompt? cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- What Makes a Good Natural Language Prompt? cites Training language models to follow instructions with human feedback
- Language Models are Few-Shot Learners cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Cited by top-tier papers4
- What Makes a Good Natural Language Prompt?Do Xuan Long, Duy Dinh, Ngoc-Hai Nguyen, Kenji Kawaguchi et al.ACL 2025 · 13 citations
- How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge AdaptationYang Zhao, Pu Wang, Hao (Frank) YangNeurIPS 2025 · 3 citations
- Less Is More: Elevating RAG via Performance-Driven Context CompressionZiqiang Cui, Yunpeng Weng, Xing Tang, Peiyang Liu et al.ICML 2026
- ACON: Optimizing Context Compression for Long-horizon LLM AgentsMinki Kang, Wei-Ning Chen, Dongge Han, Huseyin Inan et al.ICML 2026
Builds on15
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
Related papers
- LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt CompressionZhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia et al.ACL Findings 2024 · 58 citations
- Leveraging Attention to Effectively Compress Prompts for Long-Context LLMsYunlong Zhao, Haoran Wu, Bo XuAAAI 2025 · 10 citations
- DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt CompressionYi Zhao, Zuchao Li, Hai Zhao, Baoyuan Qi et al.ACL 2025 · 7 citations
- Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM InferenceBarys Liskavets, Maxim Ushakov, Shuvendu Roy, Mark Klibanov et al.AAAI 2025 · 41 citations
- Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and TransferabilityTsz Ting Chung, Leyang Cui, Lemao Liu, Xinting Huang et al.EMNLP Findings 2024 · 1 citation
