Seeing More, Saying More: Lightweight Language Experts are Dynamic Video Token Compressors

Recent advances in large video-language models have revolutionized video understanding tasks. However, their efficiency is greatly constrained by processing high volumes of visual tokens. Existing token compression strategies apply a fixed compression ratio, ignoring varying semantic density across video clips. Consequently, this leads to inadequate representation of information-rich clips due to insufficient tokens and unnecessary computation on static or content-poor ones. To address this, we propose LangDC, a Language-aware Dynamic Token Compressor. LangDC leverages a lightweight language model to describe video clips, converting them into soft caption tokens as visual representations. Trained with our proposed semantic density-aware supervision, LangDC aims to 1) cover key visual cues necessary for downstream task reasoning and 2) dynamically adjust compression ratios based on scene richness, reflected by description length. Our design mimics how humans dynamically express what they see: complex scenes (seeing more) elicit more detailed language to convey nuances (saying more), whereas simpler scenes are described with fewer words. Experimental results show that our method reduces FLOPs by 49% compared to VideoGPT+ while maintaining competitive performance. Furthermore, qualitative results demonstrate our approach adaptively adjusts the token compression ratio based on video segment richness. Codes are available at https://github.com/NIneeeeeem/LangDC .