Lune

ACL Findings2024

Understanding Fine-grained Distortions in Reports of Scientific Findings

Amelie Wührl, Dustin Wright, Roman Klinger, Isabelle Augenstein

Abstract

Distorted science communication harms individuals and society as it can lead to unhealthy behavior changes and decrease trust in scientific institutions. Given the rapidly increasing volume of science communication in recent years, a fine-grained understanding of how findings from scientific publications are reported to the general public, and methods to detect distortions from the original work automatically, are crucial. Prior work focused on individual aspects of distortions or worked with unpaired data. In this work, we make three foundational contributions towards addressing this problem: (1) annotating 1,600 instances of scientific findings from academic papers paired with corresponding findings as reported in news articles and tweets with respect to four characteristics: causality, certainty, generality and sensationalism; (2) establishing baselines for automatically detecting these characteristics; and (3) analyzing the prevalence of changes in these characteristics in both human-annotated and large-scale unlabeled data. Our results show that scientific findings frequently undergo subtle distortions when reported. Tweets distort findings more often than science news reports. Detecting fine-grained distortions automatically poses a challenging task. In our experiments, fine-tuned task-specific models consistently outperform few-shot Large Language Model prompting.

The dataset is available at

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Counterfactual LLM-based Framework for Measuring Rhetorical Style cites Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications
  • Fact or Fiction: Verifying Scientific Claims cites S2ORC: The Semantic Scholar Open Research Corpus
  • Scientific Fact-Checking: A Survey of Resources and Approaches cites Fact or Fiction: Verifying Scientific Claims
  • Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation cites Missci: Reconstructing Fallacies in Misrepresented Science
  • Missci: Reconstructing Fallacies in Misrepresented Science cites Fact or Fiction: Verifying Scientific Claims
  • Missci: Reconstructing Fallacies in Misrepresented Science cites Scientific Fact-Checking: A Survey of Resources and Approaches
  • Missci: Reconstructing Fallacies in Misrepresented Science cites Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Beyond the Crowd: LLM-Augmented Community Notes for Governing Health MisinformationJiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li et al.ACL 2026 · 17 citations
  2. Counterfactual LLM-based Framework for Measuring Rhetorical StyleJingyi Qiu, Hong Chen, Zongyi LiICLR 2026 · 1 citation
  3. Missci: Reconstructing Fallacies in Misrepresented ScienceMax Glockner, Yufang Hou, Preslav Nakov, Iryna GurevychACL 2024 · 1 citation

Builds on6

  1. S2ORC: The Semantic Scholar Open Research CorpusKyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney et al.ACL 2020 · 424 citations
  2. Writing Strategies for Science Communication: Data and Computational AnalysisTal August, Lauren Kim, Katharina Reinecke, Noah A. SmithEMNLP 2020 · 22 citations
  3. Scientific Fact-Checking: A Survey of Resources and ApproachesJuraj Vladika, Florian MatthesACL Findings 2023 · 20 citations
  4. Measuring Sentence-Level and Aspect-Level (Un)certainty in Science CommunicationsJiaxin Pei, David JurgensEMNLP 2021 · 17 citations
  5. Fact or Fiction: Verifying Scientific ClaimsDavid Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang et al.EMNLP 2020 · 6 citations

Related papers

  1. Modeling Information Change in Science Communication with Semantically Matched ParaphrasesDustin Wright, Jiaxin Pei, David Jurgens, Isabelle AugensteinEMNLP 2022 · 2 citations
  2. Semi-Supervised Exaggeration Detection of Health Science Press ReleasesDustin Wright, Isabelle AugensteinEMNLP 2021 · 7 citations
  3. Understanding News Consumers' Perceptions of Believability: A Study of Real and Fake NewsMostofa Najmus Sakib, Md Shoaib Ahmed, Francesca Spezzano, Anne HambyCSCW 2025 · 1 citation
  4. 'Don't Get Too Technical with Me': A Discourse Structure-Based Framework for Automatic Science JournalismRonald Cardenas, Bingsheng Yao, Dakuo Wang, Yufang HouEMNLP 2023 · 2 citations
  5. Pinpointing Fine-Grained Relationships between Hateful Tweets and RepliesAbdullah Albanyan, Eduardo BlancoAAAI 2022 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines