MT-RAIG: Novel Benchmark and Evaluation Framework for Retrieval-Augmented Insight Generation over Multiple Tables
Kwangwook Seo, Donguk Kwon, Dongha Lee
Abstract
Recent advancements in table-based reasoning have expanded beyond factoid-level QA to address insight-level tasks, where systems should synthesize implicit knowledge in the table to provide explainable analyses. Although effective, existing studies remain confined to scenarios where a single gold table is given alongside the user query, failing to address cases where users seek comprehensive insights from multiple unknown tables. To bridge these gaps, we propose MT-RAIG BENCH, designed to evaluate systems on Retrieval-Augmented Insight Generation over Multiple-Tables. Additionally, to tackle the suboptimality of existing automatic evaluation methods in the table domain, we further introduce a fine-grained evaluation framework MT-RAIG EVAL, which achieves better alignment with human quality judgments on the generated insights. We conduct extensive experiments and reveal that even frontier LLMs still struggle with complex multi-table reasoning, establishing our MT-RAIG BENCH as a challenging testbed for future research 1 .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext c7f959f1-d333-4a3f-83d4-866d01cefbb6Cited by top-tier papers4
- From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgeDawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi et al.EMNLP 2025 · 37 citations
- TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular ReasoningJiaru Zou, Soumya Roy, Vinay Kumar Verma, Ziyi Wang et al.ICLR 2026 · 13 citations
- P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic ChecklistKwangwook Seo, Dongha LeeACL 2026 · 2 citations
- IPQA: A Benchmark for Core Intent Identification in Personalized Question AnsweringJieyong Kim, Maryam Amirizaniani, Soojin Yoon, Dongha LeeSIGIR 2026
Builds on14
- Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
- TabFact: A Large-scale Dataset for Table-based Fact VerificationWenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang et al.ICLR 2020 · 674 citations
- Long-form factuality in large language modelsJerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu et al.NeurIPS 2024 · 182 citations
- FLASK: Fine-grained Language Model Evaluation based on Alignment Skill SetsSeonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang et al.ICLR 2024 · 176 citations
- Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations
Related papers
- MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning EvaluationXiaoyuan Li, Keqin Bao, Yubo Ma, Moxin Li et al.ACL 2026 · 12 citations
- Are We on the Right Way to Assess Document Retrieval-Augmented Generation?Wenxuan Shen, Mingjia Wang, Yaochen Wang, Dongping Chen et al.AAAI 2026
- MMTableBench: A Multi-level Multimodal Benchmark for Reasoning and Layout Complexity in Table QAXianjie Wu, Xiaohang Xu, Tingyu Jiang, Jian Yang et al.WWW 2026 · 3 citations
- MMQA: Evaluating LLMs with Multi-Table Multi-Hop Complex QuestionsJian Wu, Linyi Yang, Dongyuan Li, Yuliang Ji et al.ICLR 2025
- MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question AnsweringTeng Lin, Yuyu Luo, Honglin Zhang, Jicheng Zhang et al.EMNLP 2025 · 2 citations
