Comparison and Analysis of Value Linking in Text-to-SQL systems: [Experiments & Analysis]

Value linking in text-to-SQL systems is the problem of associating values mentioned or implied within a NL query with their database entries. Despite existing efforts, their performance and comparative effectiveness remain unexplored. This work provides the first systematic study focused exclusively on value linking. First, we present a taxonomy that encapsulates the key steps and design choices for value linking. To enable a thorough analysis, we introduce VLD-Bench, a new, challenging benchmark that incorporates 18 categories of lexical and semantic discrepancies between values referenced in the queries and database entries. Our evaluation shows that state-of-the-art text-to-SQL systems suffer up to a 64% drop in execution accuracy due to value linking errors. A fine-grained analysis reveals how design choices impact robustness to linguistic variation and system latency, offering key insights and directions for future research.