ACL Findings2024
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation
Pablo Messina, René Vidal, Denis Parra, Alvaro Soto, Vladimir Araujo
Abstract
Advancing representation learning in specialized fields like medicine remains challenging due to the scarcity of expert annotations for text and images. To tackle this issue, we present a novel two-stage framework designed to extract high-quality factual statements from freetext radiology reports in order to improve the representations of text encoders and, consequently, their performance on various downstream tasks. In the first stage, we propose a Fact Extractor that leverages large language models (LLMs) to identify factual statements from well-curated domain-specific datasets. In the second stage, we introduce a Fact Encoder (CXRFE) based on a BERT model fine-tuned with objective functions designed to improve its representations using the extracted factual data. Our framework also includes a new embedding-based metric (CXRFEScore) for evaluating chest X-ray text generation systems, leveraging both stages of our approach. Extensive evaluations show that our fact extractor and encoder outperform current state-of-the-art methods in tasks such as sentence ranking, natural language inference, and label extraction from radiology reports. Additionally, our metric proves to be more robust and effective than existing metrics commonly used in the radiology report generation literature. The code of this project is available at https://github. com/PabloMessina/CXR-Fact-Encoder. CXRFEScore: 1.000 RadGraph F1 Full: 0.750 CheXpert Acc: 1.0 CheXbert Acc: 1.0 new PICC line on the right. PICC line tip projecting in the mediastinum. potential arterial location crossing the midline. repeat PA and lateral radiograph taken approximately an hour after the previous radiograph. PICC line observed in the mid SVC. potential small right pleural effusion. stable moderate cardiomegaly CXRFEScore: 0.891 RadGraph F1 Full: 0.899 CheXpert Acc: 1.0 CheXbert Acc: 1.0 new PICC line on the right. PICC line tip in the mediastinum. appears to cross the midline. concern for potential arterial location. PA radiograph. lateral radiograph. PICC line in the mid SVC. potential small right pleural effusion. stable cardiomegaly CXRFEScore: 0.966 RadGraph F1 Full: 0.813 CheXpert Acc: 1.0 CheXbert Acc: 1.0 CheXbert CheXpert labeler Chest ImaGenome the heart is enlarged. the cardiomediastinal silhouette is enlarged. no focal consolidation. the lungs are free of focal airspace disease. no atelectasis. a device is seen. pleural effusion is seen. no fibrosis. no pneumonia. no pneumothorax is seen. no pulmonary edema. no pulmonary nodules or mass lesions identified. no fracture is seen CXRFEScore: 0.481 RadGraph F1 Full: 0.017 CheXpert Acc: 1.0 CheXbert Acc: 1.0 the heart is enlarged. the cardiomediastinal silhouette is enlarged. no focal consolidation. the lungs are free of focal airspace disease. no atelectasis. a device is seen. pleural effusion is seen. no fibrosis. no pneumonia. no pneumothorax is seen. no pulmonary edema. no pulmonary nodules or mass lesions identified. no fracture is seen CXRFEScore: 0.481 RadGraph F1 Full: 0.017 CheXpert Acc: 1.0 CheXbert Acc: 1.0 enlarged cardiac silhouette in cardiac silhouette. abnormal cardiac silhouette. picc in left shoulder. picc in mediastinum. lung opacity in right costophrenic angle. pleural effusion in right costophrenic angle. abnormal right costophrenic angle. lung opacity in right lung. pleural effusion in right lung. abnormal right lung. picc in right shoulder. picc in svc. enlarged cardiac silhouette. lung opacity. pleural effusion. picc CXRFEScore: 0.660 RadGraph F1 Full: 0.121 CheXpert Acc: 1.0 CheXbert Acc: 1.0
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports cites BERTScore: Evaluating Text Generation with BERT
- Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards cites Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
- Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards cites BERTScore: Evaluating Text Generation with BERT
Cited by top-tier papers2
- CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report GenerationPablo Messina, Andrés Villa, Juan Leon Alcazar, Karen Sanchez et al.CVPR 2026 · 1 citation
- Beyond Surface Features: Advancing Medical Vision-Language Alignment via Dynamic Evidence-Guided Preference OptimizationZixuan Huang, Zhihong Zhu, Xiaolong Liu, Yanchao Hao et al.ACL 2026
Builds on10
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- LinkBERT: Pretraining Language Models with Document LinksMichihiro Yasunaga, Jure Leskovec, Percy LiangACL 2022 · 463 citations
- Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation LearningFuying Wang, Yuyin Zhou, Shujun Wang, Varut Vardhanabhuti et al.NeurIPS 2022 · 302 citations
- Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology ReportsYuhao Zhang, Derek Merck, Emily Bao Tsai, Christopher D. Manning et al.ACL 2020 · 160 citations
- Improving the Factual Correctness of Radiology Report Generation with Semantic RewardsJean-Benoit Delbrouck, Pierre J. Chambon, Christian Bluethgen, Emily Bao Tsai et al.EMNLP Findings 2022 · 64 citations
Related papers
- Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTAkshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek et al.EMNLP 2020 · 212 citations
- Automated Structured Radiology Report GenerationJean-Benoit Delbrouck, Justin Xu, Johannes Moll, Alois Thomas et al.ACL 2025
- OraPO: Oracle-educated Reinforcement Learning for Data-efficient and Factual Radiology Report GenerationZhuoxiao Chen, Hongyang Yu, Ying Xu, Yadan Luo et al.CVPR 2026 · 3 citations
- CREPE: Rapid Chest X-ray Report Evaluation by Predicting Multi-category Error CountsGihun Cho, Seunghyun Jang, Hanbin Ko, Inhyeok Baek et al.EMNLP 2025
- FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation ModelsAlice Heiman, Xiaoman Zhang, Emma Chen, Sung Eun Kim et al.CVPR 2025
