QueryStream: Advancing Streaming Video Understanding with Query-Aware Pruning and Proactive Response
Kairui Zhang, Zhenyu Yang, Bing Wang, Shengsheng Qian, Changsheng Xu
摘要
The increasing demand for real-time interaction in online video scenarios necessitates a new class of efficient streaming video understanding models. However, existing approaches often rely on a query-agnostic ''change-is-important'' assumption, which conflates visual dynamics with semantic relevance, leading to computational redundancy and mistimed responses. To address this, we propose QueryStream, a novel framework that integrates query-awareness into the core of video processing and response scheduling. QueryStream features two synergistic components: (1) Query-Aware Differential Pruning (QDP), a policy that filters the token stream by jointly assessing semantic relevance to the query and temporal novelty against a dynamically smoothed history; and (2) Relevance-Triggered Active Response (RTAR), a dual-gated mechanism that schedules responses based on both high query relevance and significant information density. As a lightweight, training-free module, QueryStream achieves state-of-the-art performance on benchmarks such as StreamingBench and OVO-Bench under moderate pruning, and matches full-token baselines while pruning over 70% of visual tokens. Notably, our pruning mechanism generalizes to offline tasks, where it serves as a context-denoising module that benefits long-form video understanding. This work not only reveals the vast semantic redundancy in video streams relative to user intent but also establishes a promising, intent-driven direction for efficient and robust online video understanding. Code is available at: https://github.com/Zhangkr2003/QueryStream.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper20
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 被引用 7,873 次
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningWenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong 等NeurIPS 2023 · 被引用 4,013 次
- Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language ModelsMuhammad Maaz, Hanoona Abdul Rasheed, Salman Khan, Fahad KhanACL 2024 · 被引用 279 次
- TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video UnderstandingShuhuai Ren, Linli Yao, Shicheng Li, Xu Sun 等CVPR 2024 · 被引用 83 次
- VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision ComputationShiwei Wu, Joya Chen, Kevin Qinghong Lin, Qimeng Wang 等NeurIPS 2024 · 被引用 78 次
相关 Paper
- FluxMem: Adaptive Hierarchical Memory for Streaming Video UnderstandingYiweng Xie, Bo He, Junke Wang, Xiangyu Zheng 等CVPR 2026 · 被引用 25 次
- QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long Video ComprehensionYongdong Luo, Wang Chen, Weizhong Huang, Shukang Yin 等AAAI 2026
- StreamRAG: Enhancing Real-Time Video Understanding with Retrieval AugmentationJunlin Xie, Quanlong Zheng, Ruifei Zhang, Kuo Wang 等CVPR 2026
- Keyframe-Oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-form Video ProcessingYudong Liu, Jingwei Sun, Yueqian Lin, Jianyi Zhang 等ICCV 2025 · 被引用 22 次
- Vista-LLM: Decoupled Query-Guided Visual Token Pruning for Efficient Long-Video Large Language ModelsZhenyu Li, Zuchao Li, Ping Wang, Lefei Zhang 等ACL 2026
