EMNLP Findings2023
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs
Yihong Liu, Haotian Ye, Leonie Weissweiler, Renhao Pei, Hinrich Schütze
Abstract
In comparative linguistics, colexification refers to the phenomenon of a lexical form conveying two or more distinct meanings. Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP. In contrast, we identify colexification patterns of more than 2,000 concepts across 1,335 languages directly from an unannotated parallel corpus. We then propose simple and effective methods to build multilingual graphs from the colexification patterns: ColexNet and ColexNet+. ColexNet's nodes are concepts and its edges are colexifications. In ColexNet+, concept nodes are additionally linked through intermediate nodes, each representing an ngram in one of 1,334 languages. We use ColexNet+ to train -------→ ColexNet+, high-quality multilingual embeddings that are well-suited for transfer learning. In our experiments, we first show that ColexNet achieves high recall on CLICS, a dataset of crosslingual colexifications. We then evaluate -------→ ColexNet+ on roundtrip translation, sentence retrieval and sentence classification and show that our embeddings surpass several transfer learning baselines. This demonstrates the benefits of using colexification as a source of information in multilingual NLP.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead cites Unsupervised Cross-lingual Representation Learning at Scale
- Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead cites Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
- A Crosslingual Investigation of Conceptualization in 1335 Languages cites CaMEL: Case Marker Extraction without Labels
Cited by top-tier papers3
- Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine TranslationDi Wu, Christof MonzEMNLP 2023 · 5 citations
- Partial Colexifications Improve Concept EmbeddingsArne Rubehn, Johann-Mattis ListACL 2025 · 2 citations
- Charting the Landscape of African NLP: Mapping Progress and Shaping the Road AheadJesujoba Oluwadara Alabi, Michael A. Hedderich, David Ifeoluwa Adelani, Dietrich KlakowEMNLP 2025
Builds on5
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
- SenseBERT: Driving Some Sense into BERTYoav Levine, Barak Lenz, Or Dagan, Ori Ram et al.ACL 2020 · 27 citations
- Glot500: Scaling Multilingual Corpora and Language Models to 500 LanguagesAyyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini et al.ACL 2023 · 14 citations
- CaMEL: Case Marker Extraction without LabelsLeonie Weissweiler, Valentin Hofmann, Masoud Jalili Sabet, Hinrich SchützeACL 2022 · 3 citations
- A Crosslingual Investigation of Conceptualization in 1335 LanguagesYihong Liu, Haotian Ye, Leonie Weissweiler, Philipp Wicke et al.ACL 2023 · 2 citations
Related papers
- Language-agnostic BERT Sentence EmbeddingFangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan et al.ACL 2022
- A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon InductionShuo Ren, Shujie Liu, Ming Zhou, Shuai MaACL 2020 · 13 citations
- Gender Bias in Multilingual Embeddings and Cross-Lingual TransferJieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang et al.ACL 2020 · 59 citations
- Zero-shot Cross-lingual Alignment for Embedding InitializationXi Ai, Zhiyong HuangACL Findings 2024 · 2 citations
- Exploring Alignment in Shared Cross-lingual SpacesBasel Mousi, Nadir Durrani, Fahim Dalvi, Majd Hawasly et al.ACL 2024 · 1 citation
