Typology-Guided Adaptation in Multilingual Models
Ndapa Nakashole
Abstract
Multilingual models often treat language diversity as a problem of data imbalance, over-looking structural variation. We introduce the Morphological Index (MoI), a typologically grounded metric that quantifies how strongly a language relies on surface morphology for noun classification. Building on MoI, we pro-pose MoI-MoE , a Mixture of Experts model that routes inputs based on morphological structure. Evaluated on 10 Bantu languages—a large, morphologically rich and underrepresented family—MoI-MoE outperforms strong baselines, improving Swahili accuracy by 14 points on noun class recognition while maintaining performance on morphology-rich languages like Zulu. These findings highlight typo-logical structure as a practical and interpretable signal for multilingual model adaptation
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 1f46e41c-610c-4887-adfd-1999d205b331Builds on6
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- GShard: Scaling Giant Models with Conditional Computation and Automatic ShardingDmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen et al.ICLR 2021 · 1,954 citations
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
- Beyond Accuracy: Behavioral Testing of NLP Models with CheckListMarco Túlio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer SinghACL 2020 · 51 citations
- AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African LanguagesMachel Reid, Junjie Hu, Graham Neubig, Yutaka MatsuoEMNLP 2021 · 12 citations
Related papers
- Empowering Cross-lingual Behavioral Testing of NLP Models with Typological FeaturesEster Hlavnova, Sebastian RuderACL 2023 · 2 citations
- Modeling Morphological Typology for Unsupervised Learning of Language MorphologyHongzhi Xu, Jordan Kodner, Mitchell Marcus, Charles YangACL 2020 · 7 citations
- KinyaBERT: a Morphology-aware Kinyarwanda Language ModelAntoine Nzeyimana, Andre Niyongabo RubungoACL 2022 · 45 citations
- Multilingual Routing in Mixture-of-ExpertsLucas Bandarkar, Chenyuan Yang, Mohsen Fayyaz, Junlin Hu et al.ICLR 2026 · 34 citations
- Sparse MoE with Language Guided Routing for Multilingual Machine TranslationXinyu Zhao, Xuxi Chen, Yu Cheng, Tianlong ChenICLR 2024 · 19 citations
