Lune

ACL Findings2025

DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens

Shaoshen Chen, Yangning Li, Zishan Xu, Yongqin Zeng, Shunlong Wu, Xinshuo Hu, Zifei Shan, Xin Su, Jiwei Tang, Yinghui Li, Hai-Tao Zheng

Abstract

Large Language Models (LLMs) face computational inefficiencies and redundant processing when handling long context inputs, prompting a focus on compression techniques. While existing semantic vector-based compression methods achieve promising performance, these methods fail to account for the intrinsic information density variations between context chunks, instead allocating soft tokens uniformly across context chunks. This uniform distribution inevitably diminishes allocation to information-critical regions. To address this, we propose Dynamic Allocation of Soft Tokens (DAST), a simple yet effective method that leverages the LLM's intrinsic understanding of contextual relevance to guide compression. DAST combines perplexity-based local information with attention-driven global information to dynamically allocate soft tokens to the informative-rich chunks, enabling effective, context-aware compression. Experimental results across multiple benchmarks demonstrate that DAST surpasses state-of-theart methods. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

7 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors cites In-context Autoencoder for Context Compression in a Large Language Model
  • Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors cites Read As Human: Compressing Context via Parallelizable Close Reading and Skimming
  • Read As Human: Compressing Context via Parallelizable Close Reading and Skimming cites Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
  • Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors cites COMI: Coarse-to-fine Context Compression via Marginal Information Gain
  • COMI: Coarse-to-fine Context Compression via Marginal Information Gain cites Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
  • Read As Human: Compressing Context via Parallelizable Close Reading and Skimming cites xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
  • Read As Human: Compressing Context via Parallelizable Close Reading and Skimming cites In-context Autoencoder for Context Compression in a Large Language Model
  • SnapKV: LLM Knows What You are Looking for Before Generation cites LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
  • COMI: Coarse-to-fine Context Compression via Marginal Information Gain cites xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
  • COMI: Coarse-to-fine Context Compression via Marginal Information Gain cites In-context Autoencoder for Context Compression in a Large Language Model
  • COMI: Coarse-to-fine Context Compression via Marginal Information Gain cites SnapKV: LLM Knows What You are Looking for Before Generation
  • LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models cites LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers7

  1. COMI: Coarse-to-fine Context Compression via Marginal Information GainJiwei Tang, Shilei Liu, Zhicheng Zhang, Yujin Yuan et al.ICLR 2026 · 17 citations
  2. Autoencoding-Free Context Compression for LLMs via Contextual Semantic AnchorsXin Liu, Runsong Zhao, Pengcheng Huang, Xinyu Liu et al.ICLR 2026 · 16 citations
  3. Read As Human: Compressing Context via Parallelizable Close Reading and SkimmingJiwei Tang, Shilei Liu, Zhicheng Zhang, Qingsong Lv et al.ACL 2026 · 10 citations
  4. Gated Differentiable Working Memory for Long-Context Language ModelingLingrui Mei, Shenghua Liu, Yiwei Wang, Yuyao Ge et al.ACL 2026 · 4 citations
  5. Exposing and Defending the Achilles' Heel of Video Mixture-of-ExpertsSongping Wang, Qinglong Liu, Yueming Lyu, Ning Li et al.ICLR 2026 · 3 citations

Builds on22

  1. SnapKV: LLM Knows What You are Looking for Before GenerationYuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh et al.NeurIPS 2024 · 1,019 citations
  2. LongLoRA: Efficient Fine-tuning of Long-Context Large Language ModelsYukang Chen, Shengju Qian, Haotian Tang, Xin Lai et al.ICLR 2024 · 254 citations
  3. In-context Autoencoder for Context Compression in a Large Language ModelTao Ge, Jing Hu, Lei Wang, Xun Wang et al.ICLR 2024 · 158 citations
  4. xRAG: Extreme Context Compression for Retrieval-augmented Generation with One TokenXin Cheng, Xun Wang, Xingxing Zhang, Tao Ge et al.NeurIPS 2024 · 156 citations
  5. LongBench: A Bilingual, Multitask Benchmark for Long Context UnderstandingYushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu et al.ACL 2024 · 94 citations

Related papers

  1. AdmTree: Compressing Lengthy Context with Adaptive Semantic TreesYangning Li, Shaoshen Chen, Yinghui Li, Yankai Chen et al.NeurIPS 2025 · 8 citations
  2. DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt CompressionYi Zhao, Zuchao Li, Hai Zhao, Baoyuan Qi et al.ACL 2025 · 7 citations
  3. Leveraging Attention to Effectively Compress Prompts for Long-Context LLMsYunlong Zhao, Haoran Wu, Bo XuAAAI 2025 · 10 citations
  4. Seeing More, Saying More: Lightweight Language Experts are Dynamic Video Token CompressorsXiangchen Wang, Jinrui Zhang, Teng Wang, Haigang Zhang et al.EMNLP 2025 · 2 citations
  5. Core Context Aware Transformers for Long Context Language ModelingYaofo Chen, Zeng You, Shuhai Zhang, Haokun Li et al.ICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines