ACL Findings2023
Know What I don't Know: Handling Ambiguous and Unknown Questions for Text-to-SQL
Bing Wang, Yan Gao, Zhoujun Li, Jian-Guang Lou
Abstract
The task of text-to-SQL aims to convert a natural language question into its corresponding SQL query within the context of relational tables. Existing text-to-SQL parsers generate a "plausible" SQL query for an arbitrary user question, thereby failing to correctly handle problematic user questions. To formalize this problem, we conduct a preliminary study on the observed ambiguous and unanswerable cases in text-to-SQL and summarize them into 6 feature categories. Correspondingly, we identify the causes behind each category and propose requirements for handling ambiguous and unanswerable questions. Following this study, we propose a simple yet effective counterfactual example generation approach that automatically produces ambiguous and unanswerable text-to-SQL examples. Furthermore, we propose a weakly supervised DTE (Detecting-Then-Explaining) model for error detection, localization, and explanation. Experimental results show that our model achieves the best result on both real-world examples and generated examples compared with various baselines. We release our data and code at: https://github.com/wbbeyourself/DTE.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
12 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Re-examining the Role of Schema Linking in Text-to-SQL cites RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
- Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation cites RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
- Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation cites On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries
- Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation cites Re-examining the Role of Schema Linking in Text-to-SQL
Cited by top-tier papers12
- Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear QueriesXinyi He, Mengyu Zhou, Xinrun Xu, Xiaojun Ma et al.AAAI 2024 · 48 citations
- SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQLYue Gong, Chuan Lei, Xiao Qin, Kapil Vaidya et al.NeurIPS 2025 · 21 citations
- Causality Meets the Table: Debiasing LLMs for Faithful TableQA via Front-Door InterventionZhen Yang, Ziwei Du, Minghan Zhang, Wei Du et al.NeurIPS 2025 · 6 citations
- AdverIntent-Agent: Adversarial Reasoning for Repair Based on Inferred Program IntentHe Ye, Aidan Z. H. Yang, Chang Hu, Yanlin Wang et al.ISSTA 2025 · 3 citations
- From Conversation to Query Execution: Benchmarking User and Tool Interactions for EHR Database AgentsGyubok Lee, Woosog Chay, Heeyoung Kwak, Yeong Hwa Kim et al.ICLR 2026 · 2 citations
Builds on8
- Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance AwarenessJeremiah Z. Liu, Zi Lin, Shreyas Padhy, Dustin Tran et al.NeurIPS 2020 · 604 citations
- Re-examining the Role of Schema Linking in Text-to-SQLWenqiang Lei, Weixin Wang, Zhixin Ma, Tian Gan et al.EMNLP 2020 · 71 citations
- On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL QueriesTianze Shi, Chen Zhao, Jordan L. Boyd-Graber, Hal Daumé III et al.EMNLP Findings 2020 · 45 citations
- Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table PerturbationXinyu Pi, Bing Wang, Yan Gao, Jiaqi Guo et al.ACL 2022 · 41 citations
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL ParsersBailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov et al.ACL 2020 · 39 citations
Related papers
- Exploring Schema Generalizability of Text-to-SQLJieyu Li, Lu Chen, Ruisheng Cao, Su Zhu et al.ACL Findings 2023 · 3 citations
- SQL-Checker: Error Detection and Labeling for Text-to-SQL with Interpretability AnalysisXingyu Ma, Xin Tian, Lingxiang Wu, Xuepeng Wang et al.WWW 2026
- S²SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL ParsersBinyuan Hui, Ruiying Geng, Lihan Wang, Bowen Qin et al.ACL Findings 2022 · 66 citations
- Error Detection for Text-to-SQL Semantic ParsingShijie Chen, Ziru Chen, Huan Sun, Yu SuEMNLP Findings 2023 · 10 citations
- Data Ambiguity Profiling for the Generation of Training ExamplesEnzo Veltri, Gilbert Badaro, Mohammed Saeed, Paolo PapottiICDE 2023 · 12 citations
