ACL Findings2026
SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation
Long Li, Zhijian Zhou, Jiangxuan Long, Peiyang Liu, Weidi Xu, Zhe Wang, Shirui Pan, Chao Qu
Abstract
Agentic Reinforcement Learning (RL) shows promise for complex tasks, but Text-to-SQL remains mostly restricted to single-turn paradigms. A primary bottleneck is the credit assignment problem. In traditional paradigms, rewards are determined solely by the final-turn feedback, which ignores the intermediate process and leads to ambiguous credit evaluation. To address this, we propose Agentic SQL, a framework featuring a universal two-tiered reward mechanism designed to provide effective trajectory-level evaluation and dense step-level signals. First, we introduce Aggregated Trajectory Reward (ATR) to resolve multi-turn credit assignment. Using an asymmetric transition matrix, ATR aggregates process-oriented scores to incentivize continuous improvement. Leveraging Lyapunov stability theory, we provide a principled motivation for this asymmetric design: under an idealized abstraction, ATR behaves like an energy dissipation operator that discourages oscillatory trajectories and favors monotonic improvement. Second, Column-Set Matching Reward (CSMR) provides immediate step-level rewards to mitigate sparsity. By executing queries at each turn, CSMR converts binary (0/1) feedback into dense [0, 1] signals based on partial correctness. Evaluations on BIRD show a 5% gain over binary-reward GRPO. Notably, our approach outperforms SOTA Arctic-Text2SQL-R1-7B on BIRD and Spider 2.0 using identical models, propelling Text-to-SQL toward a robust multi-turn agent paradigm.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on17
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang et al.NeurIPS 2025 · 1,109 citations
- Fine-Grained Human Feedback Gives Better Rewards for Language Model TrainingZeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri et al.NeurIPS 2023 · 516 citations
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang et al.ICLR 2026 · 406 citations
- SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement LearningPeixian Ma, Xialie Zhuang, Chengjin Xu, Xuhui Jiang et al.NeurIPS 2025 · 94 citations
- OmniSQL: Synthesizing High-quality Text-to-SQL Data at ScaleHaoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang et al.VLDB 2025 · 90 citations
Related papers
- SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQLHarper Hua, Zhen Han, Zhengyuan Shen, Meng-Chieh Lee et al.ACL 2026 · 3 citations
- MARS-SQL: A Multi-Agent Reinforcement Learning Framework For Text-To-SQLHaolin Yang, Jipeng Zhang, Zhitao He, Alexander Zhou et al.ICML 2026 · 12 citations
- APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQLBowen Cao, Weibin Liao, Yushi Sun, Dong Fang et al.KDD 2026 · 7 citations
- Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise RewardHan Weng, Puzhen Wu, Longjie Cui, Yi Zhan et al.EMNLP Findings 2025 · 1 citation
- Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLZhewei Yao, Guoheng Sun, Lukasz Borchmann, Zheyu Shen et al.ACL Findings 2026 · 4 citations
