Lune

EMNLP Findings2025

FinGrAct: A Framework for FINe-GRrained Evaluation of ACTionability in Explainable Automatic Fact-Checking

Islam Eldifrawi, Shengrui Wang, Amine Trabelsi

Abstract

The field of explainable Automatic Fact-Checking (AFC) aims to enhance the transparency and trustworthiness of automated factverification systems by providing clear and comprehensible explanations.However, the effectiveness of these explanations depends on their actionability-the extent to which an AFC explanation pinpoints the error, supplies the correct fact, and backs it with sources.Despite actionability being critical for high-quality explanations, no prior research has proposed a method to evaluate it.This paper introduces FinGrAct, a fine-grained evaluation framework that can access the web and is designed to assess actionability in AFC explanations through well-defined criteria.We also introduce a novel dataset to evaluate actionability in AFC explanations.FinGrAct surpasses state-of-theart (SOTA) evaluators, achieving the highest Pearson and Kendall correlation with human judgments while demonstrating the lowest egocentric bias, making it a more robust evaluation approach for actionability evaluation in AFC. Label: False Explanation 1 (full actionability):The claim has two errors in earth's description.The errors are in the words 'flat' and 'red'.The correct version of the claim is : "Earth is round and blue".Check NASA at:

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization cites G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment
  • Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models cites BLEURT: Learning Robust Metrics for Text Generation
  • Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models cites FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. G-Eval: NLG Evaluation using Gpt-4 with Better Human AlignmentYang Liu, Dan Iter, Yichong Xu, Shuohang Wang et al.EMNLP 2023 · 549 citations
  2. Prometheus: Inducing Fine-Grained Evaluation Capability in Language ModelsSeungone Kim, Jamin Shin, Yejin Choi, Joel Jang et al.ICLR 2024 · 468 citations
  3. FLASK: Fine-grained Language Model Evaluation based on Alignment Skill SetsSeonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang et al.ICLR 2024 · 176 citations
  4. Large Language Models are Not Yet Human-Level Evaluators for Abstractive SummarizationChenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You et al.EMNLP Findings 2023 · 52 citations
  5. BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations

Related papers

  1. Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and ApproachesIslam Eldifrawi, Shengrui Wang, Amine TrabelsiACL 2024 · 4 citations
  2. Generating Media Background Checks for Automated Source Critical ReasoningMichael Sejr SchlichtkrullEMNLP Findings 2024 · 2 citations
  3. FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question AnsweringAnku Rani, S. M. Towhidul Islam Tonmoy, Dwip Dalal, Shreya Gautam et al.ACL 2023 · 16 citations
  4. Interpretable Automatic Fine-grained Inconsistency Detection in Text SummarizationHou Pong Chan, Qi Zeng, Heng JiACL Findings 2023 · 8 citations
  5. Explainable Automated Fact-Checking for Public Health ClaimsNeema Kotonya, Francesca ToniEMNLP 2020 · 10 citations
Dusk over the sea between two cliffs drawn in fine vertical lines