Unlocking SLM Potential for Data Analysis Code Generation via Non-Parametric Knowledge Distillation

Knowledge distillation from Large Language Models (LLMs) to locally hosted Small Language Models (SLMs) provides advantages for Data Analysis Code Generation (DACG) such as privacy protection. However, achieving effective distillation without resource-intensive training is challenging. This paper investigates whether LLMs can distill knowledge to SLMs through In-Context Learning (ICL), a training-free method for rapid task adaptation. We present the D AR GO : D istillation and A daptive R easoning-G uided O rchestration framework, which facilitates automatic knowledge distillation from LLMs to SLMs. D AR GO consists of three phases: exploration through an Model Orchestration Interface (MOI) , Memory Collection of successful trajectories, and Knoweldge-driven Inference . We evaluate D AR GO on three challenging DACG benchmarks (W IKI TQ, T AB MWP, and B IRD -SQL), each with in-domain training sets that enable detailed analysis of knowledge distillation effectiveness. D AR GO demonstrates a substantial relative performance improvement of 27.5% on average for the student SLMs. To further observe generalization capabilities, we evaluate the D AR G O across different teacher-student model combinations, knowledge transfer scenarios, and unified memory approaches for more advanced, test-only data analysis tasks. Our findings contribute a novel perspective on distillation methods that enhance performance for SLMs while avoiding intensive fine-tuning. The source code is available: https://github.com/accpatrick/DarGO .