EMNLP Findings2023
ECHo: A Visio-Linguistic Dataset for Event Causality Inference via Human-Centric Reasoning
Yuxi Xie, Guanzhen Li, Min-Yen Kan
Abstract
We introduce ECHO (Event Causality Inference via Human-Centric Reasoning), a diagnostic dataset of event causality inference grounded in visio-linguistic social scenarios. ECHO employs real-world human-centric deductive information built on a television crime drama. ECHO requires the Theory-of-Mind (ToM) ability to understand and reason about social interactions based on multimodal information. Using ECHO, we propose a unified Chain-of-Thought (CoT) framework to assess the reasoning capability of current AI systems. Our ToM-enhanced CoT pipeline accommodates various large foundation models in both zero-shot and few-shot visio-linguistic reasoning. We use this framework to scrutinize recent large foundation models such as InstructGPT and MiniGPT-4 on three diagnostic humancentric tasks. Further analysis demonstrates ECHO as a challenging dataset to expose imperfections and inconsistencies in reasoning. Our data and code are publicly available at https://github.com/YuxiXie/ECHo.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
- Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
- Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Builds on23
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
Related papers
- CELLO: Causal Evaluation of Large Vision-Language ModelsMeiqi Chen, Bo Peng, Yan Zhang, Chaochao LuEMNLP 2024 · 4 citations
- Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-MindMo Yu, Qiujing Wang, Shunchi Zhang, Yisi Sang et al.ICML 2024 · 22 citations
- Generating Effective CoT Traces for Mitigating Causal HallucinationYiheng Zhao, Jun YanACL Findings 2026
- Grounded Chain-of-Thought for Multimodal Large Language ModelsQiong Wu, Xiangcong Yang, Yiyi Zhou, Chenxin Fang et al.CVPR 2026 · 55 citations
- Video-Only ToM: Enhancing Theory of Mind in Multimodal Large Language ModelsSiqi Liu, Xinyang Li, Bochao Zou, Junbao Zhuo et al.CVPR 2026
