EMNLP Findings2024
MAVEN-FACT: A Large-scale Event Factuality Detection Dataset
Chunyang Li, Hao Peng, Xiaozhi Wang, Yunjia Qi, Lei Hou, Bin Xu, Juanzi Li
Abstract
Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge. However, due to the lack of highquality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community. To address these issues and provide faithful event understanding, we introduce MAVEN-FACT, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-FACT includes factuality annotations of 112, 276 events, making it the largest EFD dataset. Extensive experiments demonstrate that MAVEN-FACT is challenging for both conventional fine-tuned models and large language models (LLMs). Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-FACT also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs. Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs. Our dataset and codes can be obtained from https: //github.com/THU-KEG/MAVEN-FACT
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- A Survey on In-context Learning cites Language Models are Few-Shot Learners
- A Survey on In-context Learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Verification Reduces Hallucination in Large Language Models cites Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
- Chain-of-Verification Reduces Hallucination in Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model cites Language Models are Few-Shot Learners
- Emergence of Linear Truth Encodings in Language Models cites Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Cited by top-tier papers3
- WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development QualityChunyang Li, Yilun Zheng, Xinting Huang, Tianqing Fang et al.ICLR 2026 · 14 citations
- Emergence of Linear Truth Encodings in Language ModelsShauli Ravfogel, Gilad Yehudai, Tal Linzen, Joan Bruna et al.NeurIPS 2025 · 12 citations
- A Multi-Level Benchmark for Causal Language Understanding in Social Media DiscourseXiaohan Ding, Kaike Ping, Buse Çarik, Eugenia Ha Rim RhoEMNLP 2025 · 1 citation
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
- A Survey on In-context LearningQingxiu Dong, Lei Li, Damai Dai, Ce Zheng et al.EMNLP 2024 · 479 citations
- Chain-of-Verification Reduces Hallucination in Large Language ModelsShehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu et al.ACL Findings 2024 · 281 citations
Related papers
- MAVEN: A Massive General Domain Event Detection DatasetXiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang et al.EMNLP 2020 · 143 citations
- MAVEN-ARG: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument AnnotationXiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng et al.ACL 2024
- MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation ExtractionXiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng et al.EMNLP 2022 · 35 citations
- Can Knowledge Editing Really Correct Hallucinations?Baixiang Huang, Canyu Chen, Xiongxiao Xu, Ali Payani et al.ICLR 2025
- Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language ModelsChaoya Jiang, Hongrui Jia, Mengfan Dong, Wei Ye et al.ACM MM 2024 · 19 citations
