Lune

CVPR Findings2026

Myopia Rectification: KV Cache Pruning for MLLMs Via Dynamic Attention Subsidy and Token Reclamation

Jiedong Zhuang, Lu Lu, Ming Dai, Jian Chen, Qiang Liu, Haoji Hu

Abstract

The computational demands of visual tokens in Multi-modal Large Language Models (MLLMs) present a significant challenge for real-time inference, particularly in long-contextual scenarios. The Key-Value cache generated by these visual tokens requires substantial GPU memory, slowing down decoding speed. While existing KV cache compression methods have made some progress in language and single-image multi-modal contexts, there has been limited exploration in video-based multi-image scenarios. To bridge this gap, we conduct an in-depth analysis of the attention mechanism in MLLMs and identify an intriguing phenomenon termed “Attention Myopia”: the model inadequately allocates attention to later images, which impedes the direct application of existing compression methods to multi-image contexts. Based on this insight, we propose Dynamic Attention Subsidy (DAS), a novel technique that adjusts attention scores to ensure a more balanced selection of critical tokens. Additionally, to preserve the integrity of visual information, we propose a reclamation mechanism called “Recycling Bin”, which compresses the discarded tokens to a minimal size. Our method achieves around 1.5× speedup on inference in decoding phase with 20% of the KV cache footprint compared to the original model, while maintaining consistent performance across over 20 diverse datasets. Crucially, our technique is plug-and-play, allowing seamless integration with existing pre-trained MLLMs without incurring any additional training costs.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
  • Visual Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
  • BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation cites Learning Transferable Visual Models From Natural Language Supervision
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Flamingo: a Visual Language Model for Few-Shot Learning cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • Flamingo: a Visual Language Model for Few-Shot Learning cites Learning Transferable Visual Models From Natural Language Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on39

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  3. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
  4. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
  5. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations

Related papers

  1. HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model InferenceBowen Zeng, Feiyang Ren, Jun Zhang, Xiaoling Gu et al.ACL 2026 · 5 citations
  2. VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference AccelerationDezhan Tu, Danylo Vashchilenko, Yuzhe Lu, Panpan XuICLR 2025
  3. Q Cache: Visual Attention Is Valuable in Less than Half of Decode Layers for Multimodal Large Language ModelJiedong Zhuang, Lu Lu, Ming Dai, Rui Hu et al.AAAI 2026
  4. AirCache: Activating Inter-Modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model InferenceKai Huang, Hao Zou, Bochen Wang, Ye Xi et al.ICCV 2025 · 1 citation
  5. Efficient Multimodal Large Language Model via Dynamic KV Cache QuantizationJiahao Fan, Chien-Ming ChenAAAI 2026
Dusk over the sea between two cliffs drawn in fine vertical lines