EMNLP Findings2023
Solving the Right Problem is Key for Translational NLP: A Case Study in UMLS Vocabulary Insertion
Bernal Jimenez Gutierrez, Yuqing Mao, Vinh Nguyen, Kin Wah Fung, Yu Su, Olivier Bodenreider
Abstract
As the immense opportunities enabled by large language models become more apparent, NLP systems will be increasingly expected to excel in real-world settings. However, in many instances, powerful models alone will not yield translational NLP solutions, especially if the formulated problem is not well aligned with the real-world task. In this work, we study the case of UMLS vocabulary insertion, an important real-world task in which hundreds of thousands of new terms, referred to as atoms, are added to the UMLS, one of the most comprehensive open-source biomedical knowledge bases (Bodenreider, 2004). Previous work aimed to develop an automated NLP system to make this time-consuming, costly, and error-prone task more efficient. Nevertheless, practical progress in this direction has been difficult to achieve due to a problem formulation and evaluation gap between research output and the real-world task. In order to address this gap, we introduce a new formulation for UMLS vocabulary insertion which mirrors the real-world task, datasets which faithfully represent it and several strong baselines we developed through re-purposing existing solutions. Additionally, we propose an effective rule-enhanced biomedical language model which enables important new model behavior, outperforms all strong baselines and provides measurable qualitative improvements to editors who carry out the UVI task. We hope this case study provides insight into the considerable importance of problem formulation for the success of translational NLP solutions. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Knowledge-Rich Self-Supervision for Biomedical Entity Linking cites Scalable Zero-shot Entity Linking with Dense Entity Retrieval
- Context-Enriched Learning Models for Aligning Biomedical Vocabularies at Scale in the UMLS Metathesaurus cites Biomedical Vocabulary Alignment at Scale in the UMLS Metathesaurus
Builds on4
- Scalable Zero-shot Entity Linking with Dense Entity RetrievalLedell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel et al.EMNLP 2020 · 336 citations
- Knowledge-Rich Self-Supervision for Biomedical Entity LinkingSheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong et al.EMNLP Findings 2022 · 30 citations
- Biomedical Vocabulary Alignment at Scale in the UMLS MetathesaurusVinh Nguyen, Hong Yung Yip, Olivier BodenreiderWWW 2021 · 29 citations
- Context-Enriched Learning Models for Aligning Biomedical Vocabularies at Scale in the UMLS MetathesaurusVinh Nguyen, Hong Yung Yip, Goonmeet Bajaj, Thilini Wijesiriwardene et al.WWW 2022 · 8 citations
Related papers
- Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language ModelsGunjan Balde, Soumyadeep Roy, Mainack Mondal, Niloy GangulyEMNLP Findings 2024 · 6 citations
- Big code != big vocabulary: open-vocabulary models for source codeRafael-Michael Karampatsis, Hlib Babii, Romain Robbes, Charles Sutton et al.ICSE 2020 · 140 citations
- Autoformulation of Mathematical Optimization Models Using LLMsNicolás Astorga, Tennison Liu, Yuanzhang Xiao, Mihaela van der SchaarICML 2025
- Guiding Large Language Models for Biomedical Entity Linking via Restrictive and Contrastive DecodingZhenxi Lin, Ziheng Zhang, Jian Wu, Yefeng Zheng et al.EMNLP Findings 2025 · 3 citations
- Can We Edit LLMs for Long-Tail Biomedical Knowledge?Xinhao Yi, Jake Lever, Kevin Bryson, Zaiqiao MengEMNLP Findings 2025
