Lune

SIGMOD2026Top-tier venue

SQLBarber: A System Leveraging Large Language Models to Generate Customized and Realistic SQL Workloads

Jiale Lao, Immanuel Trummer

Abstract

Database research and development often require a large number of SQL queries for benchmarking purposes. However, acquiring real-world SQL queries is challenging due to privacy concerns, and existing generation methods offer limited options for customization and for satisfying realistic constraints. To address this issue, we present SQLBarber, a system based on Large Language Models (LLMs) to generate customized and realistic SQL workloads. SQLBarber (1) eliminates the need for users to manually craft SQL templates in advance, while providing the flexibility to accept natural language specifications to constrain SQL templates, (2) scales efficiently to generate large volumes of queries matching any user-defined cost distribution (e.g., cardinality and execution plan cost), and (3) uses execution statistics from production environments to extract SQL template specifications and query cost distributions that reflect real-world query characteristics. SQLBarber introduces (1) a declarative interface for users to effortlessly generate customized SQL templates, (2) an LLM-powered pipeline augmented with a self-correction module that profiles, refines, and prunes SQL templates based on query costs, and (3) a Bayesian Optimizer to efficiently explore predicate values and identify a set of queries that satisfy the target cost distribution. We construct and open-source ten benchmarks of varying difficulty levels and target query cost distributions based on real-world statistics from Snowflake and Amazon Redshift. Extensive experiments on these benchmarks show that SQLBarber is the only system that can generate customized SQL templates. It reduces query generation time by one to two orders of magnitude and significantly improves alignment with the target cost distribution, compared with existing methods.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Redbench: Workload Synthesis From Cloud Traces cites Building An Elastic Query Engine on Disaggregated Storage
  • Toward Drift-Aware Database Benchmarking cites Redbench: Workload Synthesis From Cloud Traces
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Redbench: Workload Synthesis From Cloud TracesJohannes Wehrstein, Roman Heinrich, Mihail Stoian, Skander Krid et al.VLDB 2026 · 7 citations
  2. Toward Drift-Aware Database BenchmarkingGuanli Liu, Renata Borovica-GajicVLDB 2026

Builds on18

  1. Building An Elastic Query Engine on Disaggregated StorageMidhul Vuppalapati, Justin Miron, Rachit Agarwal, Dan Truong et al.NSDI 2020 · 142 citations
  2. CodeS: Towards Building Open-source Language Models for Text-to-SQLHaoyang Li, Jing Zhang, Hanbing Liu, Ju Fan et al.SIGMOD 2024 · 124 citations
  3. LongBench: A Bilingual, Multitask Benchmark for Long Context UnderstandingYushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu et al.ACL 2024 · 94 citations
  4. CodexDB: Synthesizing Code for Query Processing from Natural Language Instructions using GPT-3 CodexImmanuel TrummerVLDB 2022 · 77 citations
  5. GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian OptimizationJiale Lao, Yibo Wang, Yufei Li, Jianping Wang et al.VLDB 2024 · 76 citations

Related papers

  1. SQLStorm: Taking Database Benchmarking into the LLM EraTobias Schmidt, Viktor Leis, Peter Boncz, Thomas NeumannVLDB 2025 · 21 citations
  2. Cracking Query Bottlenecks: Towards Efficiency-Oriented Text-to-SQL GenerationLi Lin, Yunfeng Shen, Lingfeng Bao, Rongxin Wu et al.ISSTA 2026
  3. PBench: Workload Synthesizer with Real Statistics for Cloud Analytics BenchmarkingYan Zhou, Chunwei Liu, Bhuvan Urgaonkar, Zhengle Wang et al.VLDB 2025 · 4 citations
  4. Text-to-SQL Empowered by Large Language Models: A Benchmark EvaluationDawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun et al.VLDB 2024 · 609 citations
  5. PRISM: Navigating Cost-Accuracy Trade-offs for NL2SQLGaurav Tarlok Kakkar, Yeounoh Chung, Fatma Özcan, Stephen Mussmann et al.SIGMOD 2026
Dusk over the sea between two cliffs drawn in fine vertical lines