Exploring Two-Phase Continual Instruction Fine-tuning for Multilingual Adaptation in Large Language Models

A key challenge for Large Language Models (LLMs) is improving their Multilingual instruction-following ability over time without deteriorating their ability in languages they already excel at, typically English. In this paper, we study a two-phase Continual Finetuning (CFT) setup toward improving a model's Multilingual adaptability. Concretely, we consider a two-phase CFT process in which an English-only end-to-end instruction fine-tuned LLM (Phase 1) is sequentially fine-tuned on a multilingual instruction dataset (Phase 2). Across MISTRAL-7B and LLAMA-3-8B and multiple dataset pairs, we show that instructional similarity between phases is critical: aligned datasets preserve or improve English while boosting multilingual ability, whereas misaligned datasets cause English degradation. We show that this degradation arises from representation shift during CFT, and that targeted mitigation strategies, including generative replay and heuristic-based layer freezing, reduce this shift and improve multilingual adaptation. IFEval (Zhou et al., 2023) and Alpaca Eval (Li et al., 2023)) and (ii) two that measure reasoning abilities (i.e., MMLU (Hendrycks et al., 2021) and HellaSwag (Zellers et al., 2019)). Likewise, we quantify a model's Multilingual Ability (MA) based on its performance on (i) two questionanswering tasks (i.e., MLQA (Lewis et al., 2019) and XQuAD (Artetxe et al., 2019)) and (ii) XLSUM (Hasan et al., 2021), a summarization task. Our Contributions CFT Outperforms Mixture. We first observe that models trained using our two-phase CFT setup