An Empirical Study of Multilingual Reasoning Distillation for Question Answering

Reasoning is one crucial capability in Large Language Models (LLMs), allowing them to perform complex tasks such as solving math problems and multi-step planning. While reasoning capability can emerge in larger models, smaller ones usually have to rely on distillation to transfer this capability from a larger model. However, recent efforts to distill reasoning capabilities have focused mainly on English, leaving multilingual distillation underexplored. To address this gap, this paper examines existing English reasoning distillation methods that utilize a variety of positive rationales in multilingual settings and proposes d-CoT-nR, which incorporates incorrect rationales as additional guidance. Empirical results from multilingual highschool examinations show that d-CoT-nR significantly surpasses the baseline, improving accuracy and the correctness of step-by-step reasoning. 1 * Work was conducted while Peerat Limkonchotiwat was a PhD candidate at VISTEC 1 https://github.com/calzonelover/d-cot-nr English CoT Multilingual Training Samples Multilingual Training Samples LLM LLM Native CoT d-CoT (Existing) d-CoT-nR (Ours)