Interactive and Explainable Region-guided Radiology Report Generation

The automatic generation of radiology reports has the potential to assist radiologists in the time-consuming task of report writing. Existing methods generate the full report from image-level features, failing to explicitly focus on anatomical regions in the image. We propose a simple yet effective region-guided report generation model that detects anatomical regions and then describes individual, salient regions to form the final report. While previous methods generate reports without the possibility of human intervention and with limited explainability, our method opens up novel clinical use cases through additional interactive capabilities and introduces a high degree of transparency and explainability. Comprehensive experiments demonstrate our method's effectiveness in report generation, outperforming previous state-of-the-art models, and highlight its interactive capabilities. The code and checkpoints are available at https://github.com/ ttanida/rgrg . * Equal contribution Region-guided Report Generation Model Generated Report The right lung is clear. Mild to moderate left pleural effusion as well as adjacent left basal atelectasis. Mild to moderate cardiomegaly. There is thoracolumbar dextroscoliosis. Left mid lung zone: Generated: Moderate left pleural effusion and moderate left lower lobe atelectasis are unchanged. Reference: Atelectasis of the left lower lobe with potential accompanying small left pleural effusion. Abdomen: Generated: NG tube tip is in the stomach. Reference: Nasogastric tube is in unchanged position. Cardiac silhouette: Generated: Moderate cardiomegaly persists. Reference: Unchanged moderate cardiomegaly. ground-truth prediction Right mid lung zone: Generated: Right lower lobe pneumonia. Reference: There is persistent subtle airspace opacity in the right mid to lower lung, this may reflect the residua of the patient's known pneumonia. Left lung: Generated: There is no pleural effusion or pneumothorax. No acute cardiopulmonary process. Reference: The left lung is clear. No pleural effusion seen.