Compositional Condition Question Answering in Tabular Understanding
Jun-Peng Jiang, Tao Zhou, De-Chuan Zhan, Han-Jia Ye
摘要
Multimodal Large Language Models (MLLMs) for tabular understanding have made significant progress in tasks such as financial report analysis and public data tests. However, our comprehensive analysis shows that these models are still limited in certain simple scenarios, particularly when handling compositional conditions in QA. Further investigation reveals that the poor performance can be attributed to two main challenges: the visual encoder's inability to accurately recognize the content of a row, and the model's tendency to overlook conditions in the question. To address these, we introduce a new Compositional Condition Tabular Understanding method, called COCOTAB. Specifically, to capture the structural relationships within tables, we enhance the visual encoder with additional row and column patches. Moreover, we introduce the conditional tokens between the visual patches and query embeddings, ensuring the model focuses on relevant parts of the table according to the conditions specified in the query. Additionally, we also introduce the Massive Multimodal Tabular Understanding (MMTU) benchmark, which comprehensively assesses the full capabilities of MLLMs in tabular understanding. Our proposed method achieves state-of-the-art performance on both existing tabular understanding benchmarks and MMTU. Our code can be available at https: //github.com/LAMDA-Tabular/MMTU.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
已有 4 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。
脉络
- LoRA: Low-Rank Adaptation of Large Language Models 引用了 Language Models are Few-Shot Learners
- TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering 引用了 VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning
- TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering 引用了 Multimodal Tabular Reasoning with Privileged Structured Information
- TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering 引用了 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning 引用了 Multimodal Tabular Reasoning with Privileged Structured Information
- Feature-aware Modulation for Learning from Temporal Tabular Data 引用了 Multimodal Tabular Reasoning with Privileged Structured Information
- Multimodal Tabular Reasoning with Privileged Structured Information 引用了 Feature-aware Modulation for Learning from Temporal Tabular Data
- Learning Transferable Visual Models From Natural Language Supervision 引用了 Language Models are Few-Shot Learners
- Multimodal Tabular Reasoning with Privileged Structured Information 引用了 Language Models are Few-Shot Learners
- Multimodal Tabular Reasoning with Privileged Structured Information 引用了 Learning Transferable Visual Models From Natural Language Supervision
- Multimodal Tabular Reasoning with Privileged Structured Information 引用了 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Multimodal Tabular Reasoning with Privileged Structured Information 引用了 Training language models to follow instructions with human feedback
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models 引用了 Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback 引用了 Language Models are Few-Shot Learners
引用它的顶会 Paper4
- Multimodal Tabular Reasoning with Privileged Structured InformationJun-Peng Jiang, Yu Xia, Hai-Long Sun, Shiyin Lu 等NeurIPS 2025 · 16 次引用
- VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning贾 子怡, Zijian Cheng, Xinyue Zhang, Kun-Yang Yu 等ICML 2026 · 3 次引用
- Feature-aware Modulation for Learning from Temporal Tabular DataHaorun Cai, Han-Jia YeNeurIPS 2025 · 3 次引用
- TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question AnsweringAn-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, Han-Jia YeICML 2026 · 1 次引用
它引用的 Paper36
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 64,255 次引用
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 47,906 次引用
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 24,707 次引用
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 22,562 次引用
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 18,833 次引用
相关 Paper
- MMTableBench: A Multi-level Multimodal Benchmark for Reasoning and Layout Complexity in Table QAXianjie Wu, Xiaohang Xu, Tingyu Jiang, Jian Yang 等WWW 2026 · 3 次引用
- TabPedia: Towards Comprehensive Visual Table Understanding with Concept SynergyWeichao Zhao, Hao Feng, Qi Liu, Jingqun Tang 等NeurIPS 2024 · 97 次引用
- Multimodal Table UnderstandingMingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She 等ACL 2024
- MMTabReal: Real-World Benchmark for Multimodal Table UnderstandingPrasham Yatinkumar Titiya, Jainil Trivedi, Chitta Baral, Vivek GuptaACL Findings 2026
- CODIS: Benchmarking Context-dependent Visual Comprehension for Multimodal Large Language ModelsFuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang 等ACL 2024 · 3 次引用
