Towards Sparse Video Understanding and Reasoning

We present REVISE (Reasoning with Video Sparsity), a multi-round agent for video question answering (VQA). Instead of uniformly sampling frames, REVISE selects a small set of informative frames, maintains a summary-as-state across rounds, and stops early when confident. It supports proprietary vision-language models (VLMs) in a "plugand-play" setting and enables reinforcement fine-tuning for open-source models. For fine-tuning, we introduce EA-GER (Evidence-Adjusted Gain for Efficient Reasoning), an annotation-free reward with three terms: (1) Confidence gain: after new frames are added, we reward the increase in the log-odds margin between the correct option and the strongest alternative; (2) Summary sufficiency: at answer time we re-ask using only the last committed summary and reward success; (3) Correct-and-early stop: answering correctly within a small turn budget is rewarded. Across multiple VQA benchmarks, REVISE improves accuracy while reducing frames, rounds, and prompt tokens, demonstrating practical sparse video reasoning. Project page: https: //sparsevideounderstanding.github.io.