A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning

Code-mixing, the interleaving of two or more languages within a sentence or discourse is ubiquitous in multilingual societies. The lack of code-mixed training data is one of the major concerns for the development of end-to-end neural network-based models to be deployed for a variety of natural language processing (NLP) applications. A potential solution is to either manually create or crowd-source the code-mixed labelled data for the task at hand, but that requires much human efforts and often not feasible because of the language specific diversity in the code-mixed text. To circumvent the data scarcity issue, we propose an effective deep learning approach for automatically generating the code-mixed text from English to multiple languages without any parallel data. In order to train the neural network, we create synthetic code-mixed texts from the available parallel corpus by modelling various linguistic properties of code-mixing. Our codemixed text generator is built upon the encoderdecoder framework, where the encoder is augmented with the linguistic and task-agnostic features obtained from the transformer based language model. We also transfer the knowledge from a neural machine translation (NMT) to warm-start the training of code-mixed generator. Experimental results and in-depth analysis show the effectiveness of our proposed code-mixed text generation on eight diverse language pairs. Language (L1) Language (L2) Code-Mixed (L1-L2) en India's agriculture is their main strength. hi भारत क कृ ष उनक मु य ताकत है । India's कृ ष इसक main strength है । en Especially valuable people like Connor Rooney. bn িবেশষত কনর িনর মেতা মূ ল বান ব ি । িবেশষ Connor Rooney মেতা valuable ব ি । en Glasses and cups, whatever they are, can be turned upside down. ta க ணா க ம ேகா ைபக , அைவ எ வாக இ தா , தைல ழாக மா றலா . Glasses ம cups அைவ எ வாக இ தா , தைல ழாக மா றலா en Democracy and development go hand in hand. de Demokratie und Entwicklung gehen Hand in Hand. Democracy und Development gehen Hand in Hand. en We abolish national embassies. fr Nous abolissons les ambassades nationales. Nous abolissons les embassies national.