ACL Findings2026
MMTabReal: Real-World Benchmark for Multimodal Table Understanding
Prasham Yatinkumar Titiya, Jainil Trivedi, Chitta Baral, Vivek Gupta
摘要
Multimodal tables i.e. tabular layouts interleaved with charts, maps, icons, and color encodings are ubiquitous in real applications yet remain difficult for Multimodal Large Language Models (MLLMs). Despite advances in text and image understanding, systematic evaluation of table-centric multimodal reasoning is limited. We introduce MMTABREAL, a MultiModal Table Benchmark, human-curated suite of 500 real-world tables paired with 4,021 question-answer pairs. MMTABREAL spans four question types, five reasoning categories, and eight structural archetypes. Evaluations of state-of-the-art models reveal substantial gaps, especially in visual grounding, spatial alignment, and multi-step inference, with 20-40% performance drops relative to existing benchmarks. These results highlight the need for architectures that more tightly fuse vision with tabular structure and support explicit numeric/logical operations. MMTABREAL is released for evaluation only, providing a rigorous, reproducible testbed that reflects the linguistic, structural, and reasoning complexity of real-world multimodal tables. Code and data are available at:https: //coral-lab-asu.github.io/mmtabreal
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
它引用的 Paper18
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu 等ICLR 2024 · 1,472 次引用
- ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical ReasoningAhmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq R. Joty 等ACL Findings 2022 · 389 次引用
- MultiModalQA: complex question answering over text, tables and imagesAlon Talmor, Ori Yoran, Amnon Catav, Dan Lahav 等ICLR 2021 · 229 次引用
- HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual DataWenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong 等EMNLP Findings 2020 · 205 次引用
- What makes unlearning hard and what to do about itKairan Zhao, Meghdad Kurmanji, George-Octavian Barbulescu, Eleni Triantafillou 等NeurIPS 2024 · 115 次引用
相关 Paper
- MMTableBench: A Multi-level Multimodal Benchmark for Reasoning and Layout Complexity in Table QAXianjie Wu, Xiaohang Xu, Tingyu Jiang, Jian Yang 等WWW 2026 · 3 次引用
- MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual SpaceAnshul Singh, Chris Biemann, Jan StrichEMNLP Findings 2025
- Knowledge-Aware Reasoning over Multimodal Semi-structured TablesSuyash Vardhan Mathur, Jainit Sushil Bafna, Kunal Kartik, Harshita Khandelwal 等EMNLP Findings 2024 · 2 次引用
- RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table AnalysisPengzuo Wu, Yuhang Yang, Guangcheng Zhu, Chao Ye 等ACL Findings 2025 · 2 次引用
- CompTab: A Comprehensive Benchmark for Real-World TableQA with Complex Reasoning and Irregular TablesZhen Yang, Wei Du, Jie Wang, Wenze Zhou 等ACL 2026
