Foreign Keys Open the Door for Faster Incremental View Maintenance
Christoforos Svingos, André Hernich, Hinnerk Gildhoff, Yannis Papakonstantinou, Yannis E. Ioannidis
Abstract
Serverless cloud-based warehousing systems enable users to create materialized views in order to speed up predictable and repeated query workloads. Incremental view maintenance (IVM) minimizes the time needed to bring a materialized view up-to-date. It allows the refresh of a materialized view solely based on the base table changes since the last refresh. In serverless cloud-based warehouses, IVM uses computations defined as SQL scripts that update the materialized view based on updates to its base tables. However, the scripts set up for materialized views with inner joins are not optimal in the presence of foreign key constraints. For instance, for a join of two tables, the state of the art IVM computations use a UNION ALL operator of two joins - one computing the contributions to the join from updates to the first table and the other one computing the remaining contributions from the second table. Knowing that one of the join keys is a foreign-key would allow us to prune all but one of the UNION ALL branches and obtain a more efficient IVM script. In this work, we explore ways of incorporating knowledge about foreign key into IVM in order to speed up its performance. Experiments in Redshift showed that the proposed technique improved the execution times of the whole refresh process up to 2 times, and up to 2.7 times the process of calculating the necessary changes that will be applied into the materialized view.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Your agent calls
Lunesearch_papers
Free to start. No credit card required.
Terminal
Install the CLIlune papers get 8f686418-c334-4908-a79a-1b998be0cb3dCited by top-tier papers1
Ask how each one uses itRelated papers
- Thrifty Query Execution via IncrementabilityDixin Tang, Zechao Shang, Aaron J. Elmore, Sanjay Krishnan et al.SIGMOD 2020 · 9 citations
- S/C: Speeding up Data Materialization with Bounded MemoryZhaoheng Li, Xinyu Pi, Yongjoo ParkICDE 2023 · 7 citations
- Tempura: A General Cost-Based Optimizer Framework for Incremental Data ProcessingZuozhi Wang, Kai Zeng, Botong Huang, Wei Chen et al.VLDB 2021 · 9 citations
- DBSP: Automatic Incremental View Maintenance for Rich Query LanguagesMihai Budiu, Tej Chajed, Frank McSherry, Leonid Ryzhyk et al.VLDB 2023 · 41 citations
- Mashup: making serverless computing useful for HPC workflows via hybrid executionRohan Basu Roy, Tirthak Patel, Vijay Gadepally, Devesh TiwariPPoPP 2022 · 53 citations
