VEU-Bench: Towards Comprehensive Understanding of Video Editing

Level Q: What Shot type does the given clip belong to? A: Over-the-shoulder shot. Recognition Q: Clarify shot motion of given video and explain grounds for the judgment. A: The shot-motion of the given video is handheld. The camera exhibits slight vertical and horizontal jitters as it follows the girl walking and then moves to focus on the two boys conversing. Q: what is the cut type used in the given video, and what is its function? A: The video uses emphasis/deemphasis cut. By cutting from medium shot of reporter, Gayle Hailstorm, to close-up of Heather in car, the emphasis cut draws attention to Heather. have made great progress in general video understanding tasks, their capabilities in video editing understanding (VEU) tasks remain unexplored. To address this gap, in this paper, we introduce VEU-Bench (Video Editing Understanding Benchmark), a comprehensive benchmark that categorizes video editing components across various dimensions, from intra-frame features like shot size to intershot attributes such as cut types and transitions. Unlike previous video editing understanding benchmarks that focus mainly on editing element classification, VEU-Bench encompasses 19 fine-grained tasks across three stages: recognition, reasoning, and judging. To enhance the annotation of VEU automatically, we built an annotation pipeline inte-grated with an ontology-based knowledge base. Through extensive experiments with 11 state-of-the-art Vid-LLMs, our findings reveal that current Vid-LLMs face significant challenges in VEU tasks, with some performing worse than random choice. To alleviate this issue, we develop Oscars 1 , a VEU expert model fine-tuned on the curated VEU-Bench dataset. It outperforms existing open-source Vid-LLMs on VEU-Bench by over 28.3% in accuracy and achieves performance comparable to commercial models like GPT-4o. We also demonstrate that incorporating VEU data significantly enhances the performance of Vid-LLMs on general video understanding benchmarks, with an average improvement of 8.3% across nine reasoning tasks.