CoLSE: A Lightweight and Robust Hybrid Learned Model for Single-Table Cardinality Estimation Using Joint CDF
Lankadinee Rathuwadu, Guanli Liu, Christopher Leckie, Renata Borovica-Gajic
Abstract
Cardinality estimation (CE), the task of predicting the result size of queries is a critical component of query optimization. Accurate estimates are essential for generating efficient query execution plans. Recently, machine learning techniques have been applied to CE, broadly categorized into query-driven and data-driven approaches. Data-driven methods learn the joint distribution of data, while query-driven methods construct regression models that map query features to cardinalities. Ideally, a CE technique should strike a balance among three key factors: accuracy, efficiency, and memory footprint. However, existing state-of-the-art models often fail to achieve this balance. To address this, we propose CoLSE, a hybrid learned approach for single-table cardinality estimation. CoLSE directly models the joint probability over queried intervals using a novel algorithm based on copula theory and integrates a lightweight neural network to correct residual estimation errors. Experimental results show that CoLSE achieves a favorable trade-off among accuracy, training time, inference latency, and model size, outperforming existing state-of-the-art methods.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 17093232-78e0-4918-808f-1b4fd5cdae0dBuilds on22
- Transformer in TransformerKai Han, An Xiao, Enhua Wu, Jianyuan Guo et al.NeurIPS 2021 · 2,148 citations
- Bao: Making Learned Query Optimization PracticalRyan Marcus, Parimarjan Negi, Hongzi Mao, Nesime Tatbul et al.SIGMOD 2021 · 242 citations
- Deep Unsupervised Cardinality EstimationZongheng Yang, Eric Liang, Amog Kamsetty, Chenggang Wu et al.VLDB 2020 · 206 citations
- Cardinality Estimation in DBMS: A Comprehensive Benchmark EvaluationYuxing Han, Ziniu Wu, Peizhi Wu, Rong Zhu et al.VLDB 2022 · 169 citations
- Are We Ready For Learned Cardinality Estimation?Xiaoying Wang, Changbo Qu, Weiyuan Wu, Jiannan Wang et al.VLDB 2021 · 156 citations
Related papers
- FACE: A Normalizing Flow based Cardinality EstimatorJiayi Wang, Chengliang Chai, Jiabin Liu, Guoliang LiVLDB 2022
- Deep Learning Models for Selectivity Estimation of Multi-Attribute QueriesShohedul Hasan, Saravanan Thirumuruganathan, Jees Augustine, Nick Koudas et al.SIGMOD 2020 · 101 citations
- Prediction Intervals for Learned Cardinality Estimation: An Experimental EvaluationSaravanan Thirumuruganathan, Suraj Shetiya, Nick Koudas, Gautam DasICDE 2022 · 7 citations
- Speeding Up End-to-end Query Execution via Learning-based Progressive Cardinality EstimationFang Wang, Xiao Yan, Man Lung Yiu, Shuai Li et al.SIGMOD 2023 · 24 citations
- Sample-Efficient Cardinality Estimation Using Geometric Deep LearningSilvan Reiner, Michael GrossniklausVLDB 2024 · 20 citations
