SSP: Self-Supervised Prompting for Cross-Lingual Transfer to Low-Resource Languages using Large Language Models

Recently, very large language models (LLMs) have shown exceptional performance on several English NLP tasks with just in-context learning (ICL), but their utility in other languages is still underexplored. We investigate their effectiveness for NLP tasks in low-resource languages (LRLs), especially in the setting of zerolabeled cross-lingual transfer (0-CLT), where no labeled training data exists for the target language but data from related medium-resource languages (MRLs) and unlabeled test data for the target language are available. We introduce Self-Supervised Prompting (SSP), a novel ICL approach tailored for the 0-CLT setting.

SSP leverages the key observation that LLMs output more accurate labels if in-context exemplars are given from the target language, even if their labels are slightly noisy. To operationalize this, since target language training data is not available in 0-CLT setup, SSP operates in two stages. In Stage I, using source MRL training data, target language's test data is noisily labeled. In Stage II, these noisy test data points are used as exemplars in ICL for further improved labeling. Additionally, our implementation of SSP uses a novel Integer Linear Programming (ILP)-based exemplar selection method that balances similarity, prediction confidence and label coverage. Experimental results on three tasks and eleven LRLs (from three regions) demonstrate that SSP strongly outperforms existing SOTA finetuned and prompting-based baselines in the 0-CLT setting.