Typology-Guided Adaptation in Multilingual Models
Ndapa Nakashole
摘要
Multilingual models often treat language diversity as a problem of data imbalance, over-looking structural variation. We introduce the Morphological Index (MoI), a typologically grounded metric that quantifies how strongly a language relies on surface morphology for noun classification. Building on MoI, we pro-pose MoI-MoE , a Mixture of Experts model that routes inputs based on morphological structure. Evaluated on 10 Bantu languages—a large, morphologically rich and underrepresented family—MoI-MoE outperforms strong baselines, improving Swahili accuracy by 14 points on noun class recognition while maintaining performance on morphology-rich languages like Zulu. These findings highlight typo-logical structure as a practical and interpretable signal for multilingual model adaptation
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper6
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- GShard: Scaling Giant Models with Conditional Computation and Automatic ShardingDmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen 等ICLR 2021 · 被引用 1,954 次
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary 等ACL 2020 · 被引用 539 次
- Beyond Accuracy: Behavioral Testing of NLP Models with CheckListMarco Túlio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer SinghACL 2020 · 被引用 51 次
- AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African LanguagesMachel Reid, Junjie Hu, Graham Neubig, Yutaka MatsuoEMNLP 2021 · 被引用 12 次
相关 Paper
- Empowering Cross-lingual Behavioral Testing of NLP Models with Typological FeaturesEster Hlavnova, Sebastian RuderACL 2023 · 被引用 2 次
- Modeling Morphological Typology for Unsupervised Learning of Language MorphologyHongzhi Xu, Jordan Kodner, Mitchell Marcus, Charles YangACL 2020 · 被引用 7 次
- KinyaBERT: a Morphology-aware Kinyarwanda Language ModelAntoine Nzeyimana, Andre Niyongabo RubungoACL 2022 · 被引用 45 次
- Multilingual Routing in Mixture-of-ExpertsLucas Bandarkar, Chenyuan Yang, Mohsen Fayyaz, Junlin Hu 等ICLR 2026 · 被引用 34 次
- Sparse MoE with Language Guided Routing for Multilingual Machine TranslationXinyu Zhao, Xuxi Chen, Yu Cheng, Tianlong ChenICLR 2024 · 被引用 19 次
