ACL Findings2024
Understanding Fine-grained Distortions in Reports of Scientific Findings
Amelie Wührl, Dustin Wright, Roman Klinger, Isabelle Augenstein
Abstract
Distorted science communication harms individuals and society as it can lead to unhealthy behavior changes and decrease trust in scientific institutions. Given the rapidly increasing volume of science communication in recent years, a fine-grained understanding of how findings from scientific publications are reported to the general public, and methods to detect distortions from the original work automatically, are crucial. Prior work focused on individual aspects of distortions or worked with unpaired data. In this work, we make three foundational contributions towards addressing this problem: (1) annotating 1,600 instances of scientific findings from academic papers paired with corresponding findings as reported in news articles and tweets with respect to four characteristics: causality, certainty, generality and sensationalism; (2) establishing baselines for automatically detecting these characteristics; and (3) analyzing the prevalence of changes in these characteristics in both human-annotated and large-scale unlabeled data. Our results show that scientific findings frequently undergo subtle distortions when reported. Tweets distort findings more often than science news reports. Detecting fine-grained distortions automatically poses a challenging task. In our experiments, fine-tuned task-specific models consistently outperform few-shot Large Language Model prompting.
The dataset is available at
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Counterfactual LLM-based Framework for Measuring Rhetorical Style cites Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications
- Fact or Fiction: Verifying Scientific Claims cites S2ORC: The Semantic Scholar Open Research Corpus
- Scientific Fact-Checking: A Survey of Resources and Approaches cites Fact or Fiction: Verifying Scientific Claims
- Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation cites Missci: Reconstructing Fallacies in Misrepresented Science
- Missci: Reconstructing Fallacies in Misrepresented Science cites Fact or Fiction: Verifying Scientific Claims
- Missci: Reconstructing Fallacies in Misrepresented Science cites Scientific Fact-Checking: A Survey of Resources and Approaches
- Missci: Reconstructing Fallacies in Misrepresented Science cites Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications
Cited by top-tier papers3
- Beyond the Crowd: LLM-Augmented Community Notes for Governing Health MisinformationJiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li et al.ACL 2026 · 17 citations
- Counterfactual LLM-based Framework for Measuring Rhetorical StyleJingyi Qiu, Hong Chen, Zongyi LiICLR 2026 · 1 citation
- Missci: Reconstructing Fallacies in Misrepresented ScienceMax Glockner, Yufang Hou, Preslav Nakov, Iryna GurevychACL 2024 · 1 citation
Builds on6
- S2ORC: The Semantic Scholar Open Research CorpusKyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney et al.ACL 2020 · 424 citations
- Writing Strategies for Science Communication: Data and Computational AnalysisTal August, Lauren Kim, Katharina Reinecke, Noah A. SmithEMNLP 2020 · 22 citations
- Scientific Fact-Checking: A Survey of Resources and ApproachesJuraj Vladika, Florian MatthesACL Findings 2023 · 20 citations
- Measuring Sentence-Level and Aspect-Level (Un)certainty in Science CommunicationsJiaxin Pei, David JurgensEMNLP 2021 · 17 citations
- Fact or Fiction: Verifying Scientific ClaimsDavid Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang et al.EMNLP 2020 · 6 citations
Related papers
- Modeling Information Change in Science Communication with Semantically Matched ParaphrasesDustin Wright, Jiaxin Pei, David Jurgens, Isabelle AugensteinEMNLP 2022 · 2 citations
- Semi-Supervised Exaggeration Detection of Health Science Press ReleasesDustin Wright, Isabelle AugensteinEMNLP 2021 · 7 citations
- Understanding News Consumers' Perceptions of Believability: A Study of Real and Fake NewsMostofa Najmus Sakib, Md Shoaib Ahmed, Francesca Spezzano, Anne HambyCSCW 2025 · 1 citation
- 'Don't Get Too Technical with Me': A Discourse Structure-Based Framework for Automatic Science JournalismRonald Cardenas, Bingsheng Yao, Dakuo Wang, Yufang HouEMNLP 2023 · 2 citations
- Pinpointing Fine-Grained Relationships between Hateful Tweets and RepliesAbdullah Albanyan, Eduardo BlancoAAAI 2022 · 9 citations
