EMNLP Findings2022
Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding
Zhichao Yang, Shufan Wang, Bhanu Pratap Singh Rawat, Avijit Mitra, Hong Yu
Abstract
Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with average length of 3,000+ tokens. This task is challenging due to a high-dimensional space of multi-label assignment (tens of thousands of ICD codes) and the long-tail challenge: only a few codes (common diseases) are frequently assigned while most codes (rare diseases) are infrequently assigned. This study addresses the long-tail challenge by adapting a prompt-based fine-tuning technique with label semantics, which has been shown to be effective under few-shot setting. To further enhance the performance in medical domain, we propose a knowledge-enhanced longformer by injecting three domain-specific knowledge: hierarchy, synonym, and abbreviation with additional pretraining using contrastive learning. Experiments on MIMIC-III-full, a benchmark dataset of code assignment, show that our proposed method outperforms previous state-of-the-art method in 14.5% in marco F1 (from 10.3 to 11.8, P<0.001). To further test our model on few-shot setting, we created a new rare diseases coding dataset, MIMIC-III-rare50, on which our model improves marco F1 from 17.1 to 30.4 and micro F1 from 17.2 to 32.6 compared to previous method.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Towards Semi-Structured Automatic ICD Coding via Tree-based Contrastive Learning cites ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network
- Towards Semi-Structured Automatic ICD Coding via Tree-based Contrastive Learning cites HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD Coding
- Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Review cites ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network
- Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Review cites HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD Coding
- RuCCoD: Towards Automated ICD Coding in Russian cites ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network
- RuCCoD: Towards Automated ICD Coding in Russian cites HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD Coding
- RuCCoD: Towards Automated ICD Coding in Russian cites Towards Semi-Structured Automatic ICD Coding via Tree-based Contrastive Learning
- RuCCoD: Towards Automated ICD Coding in Russian cites Biomedical Entity Representations with Synonym Marginalization
- Multi-Label Few-Shot ICD Coding as Autoregressive Generation with Prompt cites HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD Coding
Cited by top-tier papers5
- Multi-Label Few-Shot ICD Coding as Autoregressive Generation with PromptZhichao Yang, Sunjae Kwon, Zonghai Yao, Hong YuAAAI 2023 · 29 citations
- Towards Semi-Structured Automatic ICD Coding via Tree-based Contrastive LearningChang Lu, Chandan K. Reddy, Ping Wang, Yue NingNeurIPS 2023 · 10 citations
- Trigger Warning Assignment as a Multi-Label Document Classification ProblemMatti Wiegmann, Magdalena Wolska, Christopher Schröder, Ole Borchardt et al.ACL 2023 · 10 citations
- RuCCoD: Towards Automated ICD Coding in RussianAlexandr Nesterov, Andrey Sakhovskiy, Ivan Sviridov, Airat Valiev et al.EMNLP 2025 · 1 citation
- Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical ReviewYidong Gan, Maciej Rybinski, Ben Hachey, Jonathan K. KummerfeldACL 2025
Builds on19
- What Makes for Good Views for Contrastive Learning?Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan et al.NeurIPS 2020 · 1,631 citations
- ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural NetworkFei Li, Hong YuAAAI 2020 · 201 citations
- Biomedical Entity Representations with Synonym MarginalizationMujeen Sung, Hwisang Jeon, Jinhyuk Lee, Jaewoo KangACL 2020 · 134 citations
- Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language ModelsRobert L. Logan IV, Ivana Balazevic, Eric Wallace, Fabio Petroni et al.ACL Findings 2022 · 133 citations
- HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD CodingPengfei Cao, Yubo Chen, Kang Liu, Jun Zhao et al.ACL 2020 · 104 citations
Related papers
- Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation MechanismTong Zhou, Pengfei Cao, Yubo Chen, Kang Liu et al.ACL 2021
- A Two-Stage Decoder for Efficient ICD CodingThanh-Tung Nguyen, Viktor Schlegel, Abhinav Ramesh Kashyap, Stefan WinklerACL Findings 2023 · 11 citations
- Effective Convolutional Attention Network for Multi-label Clinical Document ClassificationYang Liu, Hua Cheng, Russell Klopfer, Matthew R. Gormley et al.EMNLP 2021 · 51 citations
- A General Knowledge Injection Framework for ICD CodingXu Zhang, Kun Zhang, Wenxin Ma, Rongsheng Wang et al.ACL Findings 2025 · 2 citations
- Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis PredictionMingyu Derek Ma, Xiaoxuan Wang, Yijia Xiao, Anthony Cuturrufo et al.AAAI 2025 · 7 citations
