Biomedical Vocabulary Alignment at Scale in the UMLS Metathesaurus

The current UMLS (Unified Medical Language System) Metathesaurus construction process for integrating over 200 biomedical source vocabularies is expensive and error-prone as it relies on the lexical algorithms and human editors for deciding if the two biomedical terms are synonymous. Recent work has aimed to improve the Metathesaurus construction process using a deep learning approach with a Siamese Network initialized with BioWordVec embeddings for predicting synonymy among biomedical terms. Recent advances in Natural Language Processing such as Transformer models like BERT and its biomedical variants such as BioBERT small, BioBERT large, BlueBERT, and SapBERT with contextualized word embeddings have achieved state-of-the-art (SOTA) performance on downstream tasks. These techniques are therefore logical candidates for a synonymy prediction task as well. In this paper, we evaluate different approaches of employing biomedical BERT-based models in two model architectures: (1) Siamese Network, and (2) Transformer for predicting synonymy in the UMLS Metathesaurus. We aim to validate if these approaches using the BERT models can actually outperform the existing approaches. In the existing Siamese Networks with LSTM and BioWordVec embeddings, we replace the BioWordVec embeddings with the biomedical BERT embeddings extracted from each BERT model using different ways of extraction. In the Transformer architecture, we evaluate the use of the different biomedical BERT models that have been pre-trained using different datasets and tasks. Given the SOTA performance of these BERT models for other downstream tasks, our experiments yield surprisingly interesting results: (1) in both model architectures, the approaches employing these biomedical BERT-based models do not outperform the existing approaches using Siamese Network with BioWordVec embeddings for the UMLS synonymy prediction task, (2) the original BioBERT large model that has not been pre-trained with the UMLS outperforms the SapBERT models that have been pre-trained with the UMLS, and (3) using the Siamese Networks yields better performance for synonymy prediction when compared to using the biomedical BERT models.