Lune

EMNLP Findings2020

Document Classification for COVID-19 Literature

Bernal Jimenez Gutierrez, Jucheng Zeng, Dongdong Zhang, Ping Zhang, Yu Su

Abstract

The global pandemic has made it more important than ever to quickly and accurately retrieve relevant scientific literature for effective consumption by researchers in a wide range of fields. We provide an analysis of several multi-label document classification models on the LitCovid dataset, a growing collection of 23,000 research papers regarding the novel 2019 coronavirus. We find that pre-trained language models fine-tuned on this dataset outperform all other baselines and that BioBERT surpasses the others by a small margin with micro-F1 and accuracy scores of around 86% and 75% respectively on the test set. We evaluate the data efficiency and generalizability of these models as essential features of any system prepared to deal with an urgent situation like the current health crisis. We perform a data ablation study to determine how important article titles are for achieving reasonable performance on this dataset. Finally, we explore 50 errors made by the best performing models on LitCovid documents and find that they often (1) correlate certain labels too closely together and (2) fail to focus on discriminative sections of the articles; both of which are important issues to address in future work. Both data and code are available on GitHub 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Related papers

  1. Detecting Health Advice in Medical Research LiteratureYingya Li, Jun Wang, Bei YuEMNLP 2021 · 1 citation
  2. DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domainsYanis Labrak, Adrien Bazoge, Richard Dufour, Mickael Rouvier et al.ACL 2023 · 19 citations
  3. Evidence-based Fact-Checking of Health-related ClaimsMourad Sarrouti, Asma Ben Abacha, Yassine Mrabet, Dina Demner-FushmanEMNLP Findings 2021 · 71 citations
  4. Fine-Tuning Language Models on Multiple Datasets for Citation Intention ClassificationZeren Shui, Petros Karypis, Daniel S. Karls, Mingjian Wen et al.EMNLP Findings 2024 · 12 citations
  5. Exploring the Impact of Corpus Diversity on Financial Pretrained Language ModelsJaeyoung Choe, Keonwoong Noh, Nayeon Kim, Seyun Ahn et al.EMNLP Findings 2023 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines