Warper: Efficiently Adapting Learned Cardinality Estimators to Data and Workload Drifts
Beibin Li, Yao Lu, Srikanth Kandula
Abstract
Recent learned cardinality estimation (CE) models are vulnerable when query predicates or the underlying datasets drift from what the models were trained upon. We propose a system Warper that accelerates model adaptation to drifts; Warper generates additional queries when limited examples are available from the new workload and carefully picks which queries to use to update the CE model. We show that Warper can be used to adapt different CE models including ones that support queries over single tables and join expressions. Experiments with different drifts suggest that Warper has a small computational cost and adapts much faster compared to state-of-the-art solutions. We also show that faster model adaptation improves query performance by shortening the period for which imperfect query plans are picked by a query optimizer due to incorrect cardinality estimates.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 0801e530-f818-4091-b0e9-848fe16d4fbcCited by top-tier papers15
- Robust Query Driven Cardinality Estimation under Changing WorkloadsParimarjan Negi, Ziniu Wu, Andreas Kipf, Nesime Tatbul et al.VLDB 2023 · 88 citations
- Breaking It Down: An In-depth Study of Index AdvisorsWei Zhou, Chen Lin, Xuanhe Zhou, Guoliang LiVLDB 2024 · 21 citations
- Eraser: Eliminating Performance Regression on Learned Query OptimizerLianggui Weng, Rong Zhu, Di Wu, Bolin Ding et al.VLDB 2024 · 19 citations
- Machine Unlearning in Learned Databases: An Experimental AnalysisMeghdad Kurmanji, Eleni Triantafillou, Peter TriantafillouSIGMOD 2024 · 12 citations
- PACE: Poisoning Attacks on Learned Cardinality EstimationJintao Zhang, Chao Zhang, Guoliang Li, Chengliang ChaiSIGMOD 2024 · 9 citations
Builds on6
- WILDS: A Benchmark of in-the-Wild Distribution ShiftsPang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie et al.ICML 2021 · 1,773 citations
- Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic SegmentationJaehoon Choi, Taekyung Kim, Changick KimICCV 2019 · 264 citations
- Deep Unsupervised Cardinality EstimationZongheng Yang, Eric Liang, Amog Kamsetty, Chenggang Wu et al.VLDB 2020 · 206 citations
- Active Learning for ML Enhanced Database SystemsLin Ma, Bailu Ding, Sudipto Das, Adith SwaminathanSIGMOD 2020 · 57 citations
- Efficiently Approximating Selectivity Functions using Low Overhead Regression ModelsAnshuman Dutt, Chi Wang, Vivek R. Narasayya, Surajit ChaudhuriVLDB 2020 · 45 citations
Related papers
- Are We Ready For Learned Cardinality Estimation?Xiaoying Wang, Changbo Qu, Weiyuan Wu, Jiannan Wang et al.VLDB 2021 · 156 citations
- Speeding Up End-to-end Query Execution via Learning-based Progressive Cardinality EstimationFang Wang, Xiao Yan, Man Lung Yiu, Shuai Li et al.SIGMOD 2023 · 24 citations
- Data-Agnostic Cardinality Learning from Imperfect WorkloadsPeizhi Wu, Rong Kang, Tieying Zhang, Jianjun Chen et al.VLDB 2025 · 1 citation
- Prediction Intervals for Learned Cardinality Estimation: An Experimental EvaluationSaravanan Thirumuruganathan, Suraj Shetiya, Nick Koudas, Gautam DasICDE 2022 · 7 citations
- Duet: Efficient and Scalable Hybrid Neural Relation UnderstandingKaixin Zhang, Hongzhi Wang, Yabin Lu, Ziqi Li et al.ICDE 2024 · 3 citations
