Where Are We? Evaluating LLM Performance on African Languages

Language policies for education across Africa. (b) Official national language policies across Africa. (c) Official regional language policies across Africa. None means no additional policy is available on a regional level. (d) We collect SAHARA to empirically evaluate LLM performance on African languages, allowing us to demonstrate with evidence how current language policies directly impact progress in the field. Figure 1: Maps of Africa showing the languages covered in this work and the language policies across the continent. The term Indigenous language is broadly defined here as one that is 'native' to the area (Walsh, 2005). The term Both in maps (a), (b), and (c) refers to Indigenous and foreign languages combined. Knowledge of what is an Indigenous language is based exclusively on Ethnologue ( https://www.ethnologue.com ).