TrustTable: A Neuro-Symbolic Auditing Framework for Faithful Table QA
Guangzhen Zhao, Dechang Kong, Tongyu Wu, Zhenjiang Dong
Abstract
Large Language Models based Table Question Answering (LLMs-based TableQA) models excel in NLP field, however, they occasionally exhibit unfaithful behavior where correct answers are derived through erroneous reasoning paths. In this condition, we propose Trust-Table, a neuro-symbolic framework designed to ensure reasoning faithfulness by auditing the reasoning processes of LLMs. Unlike monolithic LLM-based auditors, TrustTable decouples the auditing operation into two orthogonal dimensions. It enforces factual grounding by executing neurally generated Pandas code against the table, and ensures logical soundness by verifying reasoning chains through a LLM-synthesized formal solver. By integrating these symbolic checks, TrustTable enables a "Label-Free Audit Loop" that systematically identifies and rectifies reasoning flaws without human supervision. In addition, we present the TrustTable-Bench, a diagnostic dataset containing diverse error categories that range from calculation discrepancies to schema misalignments. This benchmark allows for a rigorous quantification of reasoning limitations. Experiments demonstrate that our symbolic audit detects reasoning flaws more accurately than advanced baselines. More broadly, TrustTable outperforms LLM judges in both majority voting with logical weighting and rejection sampling with process supervision.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on13
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought PromptingMiles Turpin, Julian Michael, Ethan Perez, Samuel R. BowmanNeurIPS 2023 · 1,792 citations
- Faith and Fate: Limits of Transformers on CompositionalityNouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li et al.NeurIPS 2023 · 728 citations
Related papers
- SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic VerificationWenyao Cui, Huaping Zhang, Yongyi Huang, Qiuchi Li et al.KDD 2026
- Accurate and Regret-Aware Numerical Problem Solver for Tabular Question AnsweringYuxiang Wang, Jianzhong Qi, Junhao GanAAAI 2025 · 12 citations
- FLARE: Faithful Logic-Aided Reasoning and ExplorationErik Arakelyan, Pasquale Minervini, Patrick S. H. Lewis, Pat Verga et al.EMNLP 2025 · 1 citation
- SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large SpreadsheetsZiwei Wang, Jiayuan Su, Mengyu Zhou, Huaxing Zeng et al.AAAI 2026 · 2 citations
- Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic VerificationChuxue Cao, Jinluan Yang, Haoran Li, Kunhao Pan et al.ICML 2026 · 3 citations
