Lune

CVPR Findings2026

V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning

Zixu Cheng, Jian Hu, Ziquan Liu, Chenyang Si, Wei Li, Shaogang Gong

Abstract

Human reasoning about video content typically follows a sequential process: establishing a spatio-temporal ("whenwhere") context before inferring "what" occurs. In contrast, current Video Large Language Models (Video-LLMs) often reverse this order, performing well on what-based video question answering while struggling with the preceding when and where grounding steps. Existing benchmarks mostly focus on "what" object or event presence, overlooking deeper relational and spatio-temporal reasoning. To address this gap, we propose a Video Spatio-Temporal Reasoning (V-STaR) benchmark, designed to evaluate how Video-LLMs integrate explicit temporal and spatial cues in reasoning. The benchmark is built upon a Reverse Spatio-Temporal Reasoning (RSTR) mechanism that enforces a human-like Chain-of-Thought by decomposing video understanding into a coarse-to-fine series of interrelated questions about the same event. To support this evaluation, we construct a dataset using a semi-automated GPT-4-based pipeline that generates CoT-style spatio-temporal reasoning questions. Comprehensive evaluations of 16 stateof-the-art Video-LLMs on V-STaR reveal significant gaps in their ability to reason about "when-where" to "what" events occur. Our analysis uncovers three key insights: (1) models often rely on static representations instead of understanding dynamic processes; (2) there is a gap between implicit knowledge and explicit reasoning; and (3) semantic biases dominate the models' predictions. These findings highlight current limitations and point toward the need for more robust and interpretable video reasoning models.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

8 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding cites MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers8

  1. Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal EvidenceJiahao Meng, Xiangtai Li, Haochen Wang, Tan Yue et al.ICML 2026 · 43 citations
  2. STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D IntelligenceZihan Liu, Zhikang Niu, Qiuyang Xiao, Zhisheng Zheng et al.ICLR 2026 · 12 citations
  3. MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment GroundingFuwen Luo, Shengfeng Lou, Chi Chen, Ziyue Wang et al.ACL 2026 · 10 citations
  4. STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement LearningJuntong Ni, Shiyu Wang, Qi He, Ming Jin et al.ACL 2026 · 8 citations
  5. ViKey: Enhancing Temporal Understanding in Videos via Visual PromptingYeonkyung Lee, Dayun Ju, Youngmin Kim, Seil Kang et al.CVPR 2026 · 3 citations

Builds on16

  1. Learning 2D Temporal Adjacent Networks for Moment Localization with Natural LanguageSongyang Zhang, Houwen Peng, Jianlong Fu, Jiebo LuoAAAI 2020 · 579 citations
  2. Span-based Localizing Network for Natural Language Video LocalizationHao Zhang, Aixin Sun, Wei Jing, Joey Tianyi ZhouACL 2020 · 279 citations
  3. TVQA+: Spatio-Temporal Grounding for Video Question AnsweringJie Lei, Licheng Yu, Tamara L. Berg, Mohit BansalACL 2020 · 173 citations
  4. MovieChat: From Dense Token to Sparse Memory for Long Video UnderstandingEnxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang et al.CVPR 2024 · 95 citations
  5. TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video UnderstandingShuhuai Ren, Linli Yao, Shicheng Li, Xu Sun et al.CVPR 2024 · 83 citations

Related papers

  1. Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering TaskSunqi Fan, Jiashuo Cui, Meng-Hao Guo, Shuojin YangNeurIPS 2025 · 15 citations
  2. VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video Reasoning?Yuanxin Liu, Kun Ouyang, Haoning Wu, Yi Liu et al.ICLR 2026 · 20 citations
  3. VTimeCoT: Thinking by Drawing for Video Temporal Grounding and ReasoningJinglei Zhang, Yuanfan Guo, Rolandos Alexandros Potamias, Jiankang Deng et al.ICCV 2025 · 4 citations
  4. CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question AnsweringMingfang Zhang, Jingjing Pan, Ashutosh Kumar, Rajat Saini et al.CVPR 2026 · 1 citation
  5. OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMsCaorui Li, Yu Chen, Yiyan Ji, Jin Xu et al.ICLR 2026 · 53 citations
Dusk over the sea between two cliffs drawn in fine vertical lines