Lune

Abstract

The availability of vast video collections and the accuracy of ML models has generated significant interest in video analytics systems. Since naively processing all frames using expensive models is impractical, researchers have proposed optimizations such as selectively using faster but less accurate models to replace or filter frames for expensive models. However, these optimizations are difficult to apply on queries with multiple predicates and models, as users must manually explore a large optimization space. Without significant systems expertise or time investment, an analyst may manually create an execution plan that is unnecessarily expensive and/or terribly inaccurate.

We propose Relational Hints , a declarative interface that allows users to suggest ML model relationships based on domain knowledge. Users can express two key relationships: when a model can replace another (CAN REPLACE) and when a model can be used to filter frames for another (CAN FILTER). We aim to design an interface to express model relationships informed by domain specific knowledge and define the constraints by which these relationships hold. We then present the VIVA video analytics system that uses relational hints to optimize SQL queries on video datasets. VIVA automatically selects and validates the hints applicable to the query, generates possible query plans using a formal set of transformations, and finds the best performance plan that meets a user's accuracy requirements. VIVA relieves users from rewriting and manually optimizing video queries as new models become available and execution environments evolve. We evaluate VIVA implemented on top of Spark and show that hints improve performance up to 16.6X without sacrificing accuracy.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

13 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Extract-Transform-Load for Video Streams cites BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
  • Extract-Transform-Load for Video Streams cites MIRIS: Fast Object Track Queries in Video
  • Vulcan: Automatic Query Planning for Live ML Analytics cites BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
  • Vulcan: Automatic Query Planning for Live ML Analytics cites MIRIS: Fast Object Track Queries in Video
  • TVM: A Tile-based Video Management Framework cites BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
  • TVM: A Tile-based Video Management Framework cites MIRIS: Fast Object Track Queries in Video
  • TVM: A Tile-based Video Management Framework cites Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics
  • Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics cites BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
  • VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building cites BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
  • VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building cites MIRIS: Fast Object Track Queries in Video
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers13

  1. Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud PlatformsGohar Irfan Chaudhry, Esha Choukse, Haoran Qiu, Íñigo Goiri et al.OSDI 2026 · 26 citations
  2. Extract-Transform-Load for Video StreamsFerdinand Kossmann, Ziniu Wu, Eugenie Lai, Nesime Tatbul et al.VLDB 2023 · 21 citations
  3. Vulcan: Automatic Query Planning for Live ML AnalyticsYiwen Zhang, Xumiao Zhang, Ganesh Ananthanarayanan, Anand P. Iyer et al.NSDI 2024 · 17 citations
  4. VOCALExplore: Pay-as-You-Go Video Data Exploration and Model BuildingMaureen Daum, Enhao Zhang, Dong He, Stephen Mussmann et al.VLDB 2023 · 7 citations
  5. TVM: A Tile-based Video Management FrameworkTianxiong Zhong, Zhiwei Zhang, Guo Lu, Ye Yuan et al.VLDB 2024 · 6 citations

Builds on15

  1. INFaaS: Automated Model-less Inference ServingFrancisco Romero, Qian Li, Neeraja J. Yadwadkar, Christos KozyrakisUSENIX ATC 2021 · 325 citations
  2. BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video AnalyticsDaniel Kang, Peter Bailis, Matei ZahariaVLDB 2020 · 103 citations
  3. MIRIS: Fast Object Track Queries in VideoFavyen Bastani, Songtao He, Arjun Balasingam, Karthik Gopalakrishnan et al.SIGMOD 2020 · 68 citations
  4. Video Pose Distillation for Few-Shot, Fine-Grained Sports Action RecognitionJames Hong, Matthew Fisher, Michaël Gharbi, Kayvon FatahalianICCV 2021 · 54 citations
  5. Jointly Optimizing Preprocessing and Inference for DNN-based Visual AnalyticsDaniel Kang, Ankit Mathur, Teja Veeramacheneni, Peter Bailis et al.VLDB 2021 · 50 citations

Related papers

  1. EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized ViewsZhuangdi Xu, Gaurav Tarlok Kakkar, Joy Arulraj, Umakishore RamachandranSIGMOD 2022 · 26 citations
  2. A Method for Optimizing Opaque Filter QueriesWenjia He, Michael R. Anderson, Maxwell Strome, Michael J. CafarellaSIGMOD 2020 · 15 citations
  3. SketchQL: Video Moment Querying with a Visual Query InterfaceRenzhi Wu, Pramod Chunduri, Ali Payani, Xu Chu et al.SIGMOD 2025 · 6 citations
  4. FiGO: Fine-Grained Query Optimization in Video AnalyticsJiashen Cao, Karan Sarkar, Ramyad Hadidi, Joy Arulraj et al.SIGMOD 2022 · 37 citations
  5. Optimizing Dataflow Systems for Scalable Interactive VisualizationJunran Yang, Hyekang Kevin Joo, Sai S. Yerramreddy, Dominik Moritz et al.SIGMOD 2024 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines