VideoLoom: A Video Large Language Model for Joint Spatial-Temporal Understanding
Jiapeng Shi, junke Wang, Zuyao You, Bo He, Zuxuan Wu
Abstract
Recent advancements in Video Large Language Models (Video LLMs) have demonstrated impressive results, yet existing approaches handle either temporal or spatial dimension in isolation, struggling in the analysis of complex events that require spatial-temporal integration. To bridge this gap, we propose VideoLoom, a unified Video LLM for joint spatial-temporal understanding. To facilitate the development of fine-grained spatial and temporal localization capabilities, we curate LoomData-8.7k, a character-centric video dataset with temporally grounded and spatially localized captions. With this, VideoLoom achieves the state-of-the-art performance across a variety of spatial and temporal benchmarks. In addition, we introduce LoomBench, a benchmark consisting of temporal, spatial, and compositional video–question pairs, with a novel metric &, enabling a comprehensive evaluation of Video LLMs from diverse aspects. Collectively, these contributions offer a universal and effective suite for joint spatial-temporal video understanding, setting a new standard in multimodal intelligence.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 21f45e28-8232-49dc-a5ec-8b01119ad308Builds on28
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- Detecting Moments and Highlights in Videos via Natural Language QueriesJie Lei, Tamara L. Berg, Mohit BansalNeurIPS 2021 · 425 citations
- Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language ModelsMuhammad Maaz, Hanoona Abdul Rasheed, Salman Khan, Fahad KhanACL 2024 · 279 citations
- MeViS: A Large-scale Benchmark for Video Segmentation with Motion ExpressionsHenghui Ding, Chang Liu, Shuting He, Xudong Jiang et al.ICCV 2023 · 242 citations
Related papers
- VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMYuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng et al.CVPR 2025
- SAMA: Towards Multi-Turn Referential Grounded Video Chat with Large Language ModelsYe Sun, Hao Zhang, Henghui Ding, Tiehua Zhang et al.NeurIPS 2025 · 9 citations
- MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkKunchang Li, Yali Wang, Yinan He, Yizhuo Li et al.CVPR 2024
- TIME: Temporal-Sensitive Multi-Dimensional Instruction Tuning and Robust Benchmarking for Video-LLMsYunxiao Wang, Meng Liu, Wenqi Liu, Xuemeng Song et al.AAAI 2026 · 1 citation
- ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded SegmentationAli Athar, Xueqing Deng, Liang-Chieh ChenCVPR 2025
