PRO-CS : An Instance-Based Prompt Composition Technique for Code-Switched Tasks

Code-switched (CS) data is ubiquitous in today's globalized world, but the dearth of annotated datasets in code-switching poses a significant challenge for learning diverse tasks across different language pairs. Parameter-efficient prompt-tuning approaches conditioned on frozen language models have shown promise for transfer learning in limited-resource setups. In this paper, we propose a novel instancebased prompt composition technique, PRO-CS, for CS tasks that combine language and task knowledge. We compare our approach with prompt-tuning and fine-tuning for codeswitched tasks on 10 datasets across 4 language pairs. Our model outperforms the prompttuning approach by significant margins across all datasets and outperforms or remains at par with fine-tuning by using just 0.18% of total parameters. We also achieve competitive results when compared with the fine-tuned model in the low-resource cross-lingual and crosstask setting, indicating the effectiveness of our approach to incorporate new code-switched tasks. Our code and models will be available at https://github.com/srijan-bansal/PRO-CS