Lune

ACL Findings2025

Exploring LLM Annotation for Adaptation of Clinical Information Extraction Models under Data-sharing Restrictions

Seiji Shimizu, Shohei Hisada, Yutaka Uno, Shuntaro Yada, Shoko Wakamiya, Eiji Aramaki

Abstract

In-hospital text data contains valuable clinical information, yet deploying fine-tuned small language models (SLMs) for information extraction remains challenging due to differences in formatting and vocabulary across institutions. Since access to the original in-hospital data (source domain) is often restricted, annotated data from the target hospital (target domain) is crucial for domain adaptation. However, clinical annotation is notoriously expensive and time-consuming, as it demands clinical and linguistic expertise. To address this issue, we leverage large language models (LLMs) to annotate the target-domain data for the adaptation. We conduct experiments on four clinical information extraction tasks, including eight targetdomain datasets. Experimental results show that LLM-annotated data consistently enhances SLM performance and, with a larger number of annotated data, outperforms manual annotation in three out of four tasks 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Supervised Knowledge Makes Large Language Models Better In-context Learners cites Small Models are Valuable Plug-ins for Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on9

  1. Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain AdaptationJian Liang, Dapeng Hu, Jiashi FengICML 2020 · 1,624 citations
  2. LLMaAA: Making Large Language Models as Active AnnotatorsRuoyu Zhang, Yanzeng Li, Yongliang Ma, Ming Zhou et al.EMNLP Findings 2023 · 60 citations
  3. Small Models are Valuable Plug-ins for Large Language ModelsCanwen Xu, Yichong Xu, Shuohang Wang, Yang Liu et al.ACL Findings 2024 · 56 citations
  4. Supervised Knowledge Makes Large Language Models Better In-context LearnersLinyi Yang, Shuibai Zhang, Zhuohao Yu, Guangsheng Bao et al.ICLR 2024 · 28 citations
  5. Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model promptingMarco Naguib, Xavier Tannier, Aurélie NévéolEMNLP Findings 2024 · 13 citations

Related papers

  1. Data-Constrained Synthesis of Training Data for De-IdentificationThomas Vakili, Aron Henriksson, Hercules DalianisACL 2025 · 3 citations
  2. Large language models are few-shot clinical information extractorsMonica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim et al.EMNLP 2022 · 285 citations
  3. Enhancing Small Medical Learners with Privacy-preserving Contextual PromptingXinlu Zhang, Shiyang Li, Xianjun Yang, Chenxin Tian et al.ICLR 2024 · 13 citations
  4. Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language ModelsRan Xu, Hejie Cui, Yue Yu, Xuan Kan et al.ACL Findings 2024 · 24 citations
  5. Expert-guided Clinical Text Augmentation via Query-Based Model CollaborationDongkyu Cho, Miao Zhang, Gregory Lyng, Rumi ChunaraICML 2026
Dusk over the sea between two cliffs drawn in fine vertical lines