Lune

EMNLP Findings2023

Ensemble-Instruct: Instruction Tuning Data Generation with a Heterogeneous Mixture of LMs

Young-Suk Lee, Md. Arafat Sultan, Yousef El-Kurdi, Tahira Naseem, Asim Munawar, Radu Florian, Salim Roukos, Ramón Fernandez Astudillo

Abstract

Using in-context learning (ICL) for data generation, techniques such as Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023) can train strong conversational agents with only a small amount of human supervision. One limitation of these approaches is that they resort to very large language models (around 175B parameters) that are also proprietary and non-public. Here we explore the application of such techniques to language models that are much smaller (around 10B-40B parameters) and have permissive licenses. We find the Self-Instruct approach to be less effective at these sizes and propose new ICL methods that draw on two main ideas: (a) Categorization and simplification of the ICL templates to make prompt learning easier for the LM, and (b) Ensembling over multiple LM outputs to help select high-quality synthetic examples. Our algorithm leverages the 175 Self-Instruct seed tasks and employs separate pipelines for instructions that require an input and instructions that do not. Empirical investigations with different LMs show that: (1) Our proposed method yields higher-quality instruction tuning data than Self-Instruct, (2) It improves performances of both vanilla and instructiontuned LMs by significant margins, and (3) Smaller instruction-tuned LMs generate more useful outputs than their larger un-tuned counterparts. Our codebase is available at https: //github.com/IBM/ensemble-instruct.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Making, Not Taking, the Best of N cites Training language models to follow instructions with human feedback
  • QLoRA: Efficient Finetuning of Quantized LLMs cites Finetuned Language Models are Zero-Shot Learners
  • QLoRA: Efficient Finetuning of Quantized LLMs cites Training language models to follow instructions with human feedback
  • WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions cites Finetuned Language Models are Zero-Shot Learners
  • WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions cites Training language models to follow instructions with human feedback
  • Finetuned Language Models are Zero-Shot Learners cites Training language models to follow instructions with human feedback
  • Training language models to follow instructions with human feedback cites Finetuned Language Models are Zero-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Masking Teacher and Reinforcing Student for Distilling Vision-Language ModelsByung-Kwan Lee, Yu-Chiang Frank Wang, Ryo HachiumaCVPR 2026 · 7 citations
  2. Making, Not Taking, the Best of NAmmar Khairi, Daniel D'souza, Marzieh Fadaee, Julia KreutzerICLR 2026 · 3 citations

Builds on13

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  3. Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
  4. Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingStella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley et al.ICML 2023 · 1,822 citations
  5. WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex InstructionsCan Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng et al.ICLR 2024 · 1,206 citations

Related papers

  1. REInstruct: Building Instruction Data from Unlabeled CorpusShu Chen, Xinyan Guan, Yaojie Lu, Hongyu Lin et al.ACL Findings 2024 · 3 citations
  2. SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-ReflectionLiangxin Liu, Xuebo Liu, Derek F. Wong, Dongfang Li et al.NeurIPS 2024 · 49 citations
  3. Review-Instruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language ModelsJiangxu Wu, Cong Wang, Tianhuang Su, Jun Yang et al.ACL Findings 2025
  4. Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with NothingZhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng et al.ICLR 2025
  5. SelfCodeAlign: Self-Alignment for Code GenerationYuxiang Wei, Federico Cassano, Jiawei Liu, Yifeng Ding et al.NeurIPS 2024 · 79 citations
Dusk over the sea between two cliffs drawn in fine vertical lines