Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation

Natural Language Processing (NLP) is becoming a dominant subset of artificial intelligence, as the need for machines to understand human language becomes increasingly indispensable. Several NLP applications are ubiquitous, partly due to the myriad datasets being churned out daily through mediums like social networking sites. However, the growing development has not been evident in most African languages due to persistent resource limitations, among other issues. Yorùbá language, a tonal and morphologically rich African language, suffers a similar fate, resulting in limited NLP usage. To encourage further research towards improving this situation, this systematic literature review aims to comprehensively analyse studies addressing NLP development for Yorùbá, identifying challenges, resources, techniques, and applications. A well-defined search string from a structured protocol was employed to search, select, and analyse 105 primary studies between 2014 and 2024 from reputable databases. The review highlights the scarcity of annotated corpora, the limited availability of pre-trained language models, and linguistic challenges like tonal complexity and diacritic dependency as significant obstacles. It also revealed the prominent techniques, including rule-based methods, statistical methods, deep learning, and transfer learning, which were implemented alongside datasets of Yorùbá speech corpora, among others. The findings reveal a growing body of multilingual and monolingual resources, even though the field is constrained by socio-cultural factors such as code-switching and the desertion of language for digital usage. This review synthesises existing research, providing a foundation for advancing NLP for Yorùbá and in African languages generally. It aims to guide future research by identifying gaps and opportunities, thereby contributing to the broader inclusion of Yorùbá and other under-resourced African languages in global NLP advancements.