Lune

ACL Findings2024

Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios

Yuhang Zhou, Wei Ai

Abstract

There is increasing interest in distilling taskspecific knowledge from large language models (LLM) to smaller student models. Nonetheless, LLM distillation presents a dual challenge: 1) there is a high cost associated with querying the teacher LLM, such as GPT-4, for gathering an ample number of demonstrations; 2) the teacher LLM might provide imperfect outputs with a negative impact on the student's learning process. To enhance sample efficiency within resource-constrained, imperfect teacher scenarios, we propose a three-component framework leveraging three signal types. The first signal is the student's self-consistency (consistency of student multiple outputs), which is a proxy of the student's confidence. Specifically, we introduce a "teaching assistant" (TA) model to assess the uncertainty of both the student's and the teacher's outputs via confidence scoring, which serves as another two signals for student training. Furthermore, we propose a two-stage training schema to first warm up the student with a small proportion of data to better utilize student's signal. Experiments have shown the superiority of our proposed framework for four complex reasoning tasks. On average, our proposed two-stage framework brings a relative improvement of up to 20.79% compared to fine-tuning without any signals across datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Capture the Key in Reasoning to Enhance CoT Distillation Generalization cites Large Language Models are Zero-Shot Reasoners
  • Capture the Key in Reasoning to Enhance CoT Distillation Generalization cites Language Models are Few-Shot Learners
  • Capture the Key in Reasoning to Enhance CoT Distillation Generalization cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Capture the Key in Reasoning to Enhance CoT Distillation Generalization cites Training language models to follow instructions with human feedback
  • Learning from Diverse Reasoning Paths with Routing and Collaboration cites Large Language Models are Zero-Shot Reasoners
  • Reflexion: language agents with verbal reinforcement learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
  • Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
  • Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. STAT: Skill-Targeted Adaptive TrainingYinghui He, Abhishek Panigrahi, Yong Lin, Sanjeev AroraICLR 2026 · 3 citations
  2. Capture the Key in Reasoning to Enhance CoT Distillation GeneralizationChengwei Dai, Kun Li, Wei Zhou, Songlin HuACL 2025
  3. Learning from Diverse Reasoning Paths with Routing and CollaborationZhenyu Lei, Zhen Tan, Song Wang, Yaochen Zhu et al.EMNLP 2025
  4. TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding DistillationQuoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi, Linh Ngo Van et al.ACL 2026
  5. Large Language Models Struggle to Describe the Haystack without Human Help: A Social Science-Inspired Evaluation of Topic ModelsZongxia Li, Lorena Calvo-Bartolomé, Alexander Miserlis Hoyle, Paiheng Xu et al.ACL 2025

Builds on26

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  3. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  4. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
  5. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations

Related papers

  1. Democratizing Reasoning Ability: Tailored Learning from Large Language ModelZhaoyang Wang, Shaohan Huang, Yuxuan Liu, Jiahai Wang et al.EMNLP 2023 · 8 citations
  2. Teach Small Models to Reason by Curriculum DistillationWangyi Jiang, Yaojie Lu, Hongyu Lin, Xianpei Han et al.EMNLP 2025
  3. On the Generalization vs Fidelity Paradox in Knowledge DistillationSuhas Kamasetty Ramesh, Ayan Sengupta, Tanmoy ChakrabortyACL Findings 2025 · 1 citation
  4. MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language ModelsJustin Chih-Yao Chen, Swarnadeep Saha, Elias Stengel-Eskin, Mohit BansalICML 2024 · 32 citations
  5. ELAD: Explanation-Guided Large Language Models Active DistillationYifei Zhang, Bo Pan, Chen Ling, Yuntong Hu et al.ACL Findings 2024 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines