Lune

ACL Findings2026

Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects

Jun Zhang, Yicheng Ji, Feiyang Ren, Yihang Li, Bowen Zeng, Zonghao Chen, Ke Chen, Lidan Shou, Gang Chen, Huan Li

Abstract

Large Vision-Language Models (LVLMs) enable sophisticated reasoning over images and videos, yet their inference is hindered by a systemic efficiency barrier known as visual token dominance.This overhead is driven by a multi-regime interplay between high-resolution feature extraction, quadratic attention scaling, and memory bandwidth constraints.We present a systematic taxonomy of efficiency techniques structured around the inference lifecycle, consisting of encoding, prefilling, and decoding.Unlike prior reviews focused on isolated optimizations, we analyze the end-to-end pipeline to reveal how upstream decisions dictate downstream bottlenecks, covering compute-bound visual encoding, the intensive prefilling of massive contexts, and the "visual memory wall" in bandwidth-bound decoding.By decoupling the efficiency landscape into the axes of shaping information density, managing long-context attention, and overcoming memory limits, this work provides a structured analysis of how isolated optimizations compose to navigate the trade-off between visual fidelity and system efficiency.The survey concludes by outlining four future frontiers supported by pilot empirical insights, including hybrid compression based on functional unit sensitivity, modality-aware decoding with relaxed verification, progressive state management for streaming continuity, and stage-disaggregated serving through hardware-algorithm codesign.Our literature repository is at https://github.com/SuDIS-ZJU/Efficien t-LVLMs-Inference. Adaptive Resolution (Sec. 3.4)Adjust the resolution of each image or frame before visual information tokenization. Encoding-Side Token CompressionPre-projection compressing guided by encoder signals to reduce tokens. Efficient Vision Encoders (Sec. 3.1) Keyframe Selection (Sec. 3.3)Select informative frames reducing redundancy while preserving semantics. Efficient Inference for LVLMs Encoding Stage ( §3)Efficient Vision Encoders ( §3.1)

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning cites MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
  • Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
  • Visual Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
  • Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism cites See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs
  • See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs cites Visual Instruction Tuning
  • Flamingo: a Visual Language Model for Few-Shot Learning cites Learning Transferable Visual Models From Natural Language Supervision
  • MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models cites Flamingo: a Visual Language Model for Few-Shot Learning
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation PerspectiveYuan Feng, Junlin Lv, Haoyu Guo, Yukun Cao et al.ICML 2026 · 21 citations
  2. Double: Breaking the Acceleration Limit via Double Retrieval Speculative ParallelismYuhao Shen, Tianyu Liu, Junyi Shen, Jinyang Wu et al.ACL 2026 · 12 citations
  3. See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMsYicheng Ji, Jun Zhang, Jinpeng Chen, Cong Wang et al.ACL 2026 · 4 citations
  4. REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache CompressionMengjie Li, Yuan Feng, Xike Xie, William J. SongACL 2026

Builds on112

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  3. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
  4. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong et al.NeurIPS 2023 · 4,013 citations
  5. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language ModelsDeyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li et al.ICLR 2024 · 3,079 citations

Related papers

  1. ZipVL: Accelerating Vision-Language Models Through Dynamic Token SparsityYefei He, Feng Chen, Jing Liu, Wenqi Shao et al.ICCV 2025 · 1 citation
  2. Reducing Token Redundancy in LVLMs: A Systematic Review of Token Pruning MethodsHanzhang Yuan, Mengxuan Hu, Wenhao Zhang, Tianlong Wang et al.ACL 2026
  3. Variation-aware Vision Token Dropping for Faster Large Vision-Language ModelsChen junjie, Xuyang Liu, Zichen Wen, Yiyu Wang et al.CVPR 2026 · 24 citations
  4. Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe PriorYulin Li, Haokun Gui, Ziyang Fan, Junjie Wang et al.NeurIPS 2025 · 18 citations
  5. Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models Via Adaptive Token SkippingWeili Zeng, Ziyuan Huang, Kaixiang Ji, Yichao YanICCV 2025
Dusk over the sea between two cliffs drawn in fine vertical lines