Towards Cost-Optimal Query Processing in the Cloud
Viktor Leis, Maximilian Kuschewski
Abstract
Public cloud providers offer hundreds of heterogeneous hardware instances. For analytical query processing systems, this presents a major challenge: depending on the hardware configuration, performance and cost may differ by orders of magnitude. We propose a simple and intuitive model that takes the workload, hardware, and cost into account to determine the optimal instance configuration. We discuss how such a model-based approach can significantly reduce costs and also guide the evolution of cloud-native database systems to achieve our vision of cost-optimal query processing.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers13
- BtrBlocks: Efficient Columnar Compression for Data LakesMaximilian Kuschewski, David Sauerwein, Adnan Alhomssi, Viktor LeisSIGMOD 2023 · 47 citations
- Exploiting Cloud Object Storage for High-Performance AnalyticsDominik Durner, Viktor Leis, Thomas NeumannVLDB 2023 · 45 citations
- Cloud Analytics BenchmarkAlexander van Renen, Viktor LeisVLDB 2023 · 32 citations
- Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data ProcessingChenghao Lyu, Qi Fan, Fei Song, Arnab Sinha et al.VLDB 2022 · 14 citations
- Cost Modelling for Optimal Data Placement in Heterogeneous Main MemoryRobert Lasch, Thomas Legler, Norman May, Bernhard Scheirle et al.VLDB 2022 · 12 citations
Builds on3
- Building An Elastic Query Engine on Disaggregated StorageMidhul Vuppalapati, Justin Miron, Rachit Agarwal, Dan Truong et al.NSDI 2020 · 142 citations
- Lambada: Interactive Data Analytics on Cold Data Using Serverless Cloud InfrastructureIngo Müller, Renato Marroquín, Gustavo AlonsoSIGMOD 2020 · 135 citations
- OPTIMUSCLOUD: Heterogeneous Configuration Optimization for Distributed Databases in the CloudAshraf Mahgoub, Alexander Medoff, Rakesh Kumar, Subrata Mitra et al.USENIX ATC 2020 · 63 citations
Related papers
- Saving Money for Analytical Workloads in the CloudTapan Srivastava, Raul Castro FernandezVLDB 2024 · 1 citation
- On-Demand State Separation for Cloud Data WarehousingChristian Winter, Jana Giceva, Thomas Neumann, Alfons KemperVLDB 2022 · 9 citations
- Costream: Learned Cost Models for Operator Placement in Edge-Cloud EnvironmentsRoman Heinrich, Carsten Binnig, Harald Kornmayer, Manisha LuthraICDE 2024 · 10 citations
- Cackle: Analytical Workload Cost and Performance Stability With Elastic PoolsMatthew Perron, Raul Castro Fernandez, David J. DeWitt, Michael J. Cafarella et al.SIGMOD 2024 · 7 citations
- Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our FindingsTarique Siddiqui, Alekh Jindal, Shi Qiao, Hiren Patel et al.SIGMOD 2020 · 80 citations
