Lune

ACL Findings2026

Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine

Sebastian Antony Joseph, Lily Chen, Barry Wei, Michael Mackert, Iain James Marshall, Paul Pu Liang, Ramez Kouzy, Byron C. Wallace, Junyi Jessy Li

Abstract

Technological progress has led to concrete advancements in tasks that were regarded as challenging, such as automatic fact-checking. Interest in adopting these systems for public health and medicine has grown due to the high-stakes nature of medical decisions and challenges in critically appraising a vast and diverse medical literature. Evidence-based medicine connects to every individual, and yet the nature of it is highly technical, rendering the medical literacy of majority users inadequate to sufficiently navigate the domain. Such problems with medical communication ripen the ground for end-to-end fact-checking agents: check a claim against current medical literature and return with an evidence-backed verdict. And yet, such systems remain largely unused.

In this position paper, developed with expert input, we present the first study examining how clinical experts verify real claims from social media by synthesizing medical evidence. In searching for this upper-bound, we reveal fundamental challenges in end-toend fact-checking when applied to medicine: Difficulties connecting claims in the wild to scientific evidence in the form of clinical trials; ambiguities in underspecified claims mixed with mismatched intentions; and inherently subjective veracity labels. We argue that factchecking should be approached as an interactive communication problem, rather than an end-to-end process. Data and code are available at https://github.com/SebaJoe/ decide-less-communicate-more.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on14

  1. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text GenerationSewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis et al.EMNLP 2023 · 225 citations
  2. True or False: Studying the Work Practices of Professional Fact-CheckersNicholas Micallef, Vivienne Armacost, Nasir D. Memon, Sameer PatilCSCW 2022 · 71 citations
  3. Human and Technological Infrastructures of Fact-checkingPrerna Juneja, Tanushree MitraCSCW 2022 · 59 citations
  4. Fact-Checking Complex Claims with Program-Guided ReasoningLiangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu et al.ACL 2023 · 45 citations
  5. FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language ModelsLiqiang Jing, Ruosen Li, Yunmo Chen, Xinya DuEMNLP Findings 2024 · 31 citations

Related papers

  1. Explainable Automated Fact-Checking for Public Health ClaimsNeema Kotonya, Francesca ToniEMNLP 2020 · 10 citations
  2. Combining Evidence and Reasoning for Biomedical Fact-CheckingMariano Barone, Antonio Romano, Giuseppe Riccio, Marco Postiglione et al.SIGIR 2025 · 3 citations
  3. Evidence-based Fact-Checking of Health-related ClaimsMourad Sarrouti, Asma Ben Abacha, Yassine Mrabet, Dina Demner-FushmanEMNLP Findings 2021 · 71 citations
  4. Rethinking the Reliability of Evidence in End-to-End Fact-Checking from the Causal PerspectiveXubo Liu, Wenya Guo, Ruxue Yan, Xumeng Liu et al.ACM MM 2025
  5. Leveraging Social Media for Medical Text SimplificationNikhil Pattisapu, Nishant Prabhu, Smriti Bhati, Vasudeva VarmaSIGIR 2020 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines