Leveraging High-Resource English Corpora for Cross-lingual Domain Adaptation in Low-Resource Japanese Medicine via Continued Pre-training

Limited low-resource language corpora in professional domains like medicine hinder crosslingual domain adaptation of pre-trained large language models (PLMs). While abundant English medical corpora could complement this scarcity, the effective mixture of English and target language, including machine-translated content, remains underexplored. We examined how corpus compositional statistics (e.g., token sizes and language proportions) affect performance on a Japanese-English medical knowledge benchmark. Through continued pretraining of a bilingual PLM on multilingual corpora with varying proportions of English and Japanese texts (both original and machinetranslated), we analyzed correlations between corpus compositional statistics and fine-grained task performance. Our findings suggest a practical approach to optimizing multilingual corpora for cross-lingual domain adaptation, which requires leveraging specialized knowledge from English corpora while ensuring sufficient coverage of language-specific expressions in a target language (Japanese). Such insights will contribute to the development of multilingual models that effectively leverage English-language resources in various professional domains with low-resource languages.