EMNLP Findings2023
Ensemble-Instruct: Instruction Tuning Data Generation with a Heterogeneous Mixture of LMs
Young-Suk Lee, Md. Arafat Sultan, Yousef El-Kurdi, Tahira Naseem, Asim Munawar, Radu Florian, Salim Roukos, Ramón Fernandez Astudillo
Abstract
Using in-context learning (ICL) for data generation, techniques such as Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023) can train strong conversational agents with only a small amount of human supervision. One limitation of these approaches is that they resort to very large language models (around 175B parameters) that are also proprietary and non-public. Here we explore the application of such techniques to language models that are much smaller (around 10B-40B parameters) and have permissive licenses. We find the Self-Instruct approach to be less effective at these sizes and propose new ICL methods that draw on two main ideas: (a) Categorization and simplification of the ICL templates to make prompt learning easier for the LM, and (b) Ensembling over multiple LM outputs to help select high-quality synthetic examples. Our algorithm leverages the 175 Self-Instruct seed tasks and employs separate pipelines for instructions that require an input and instructions that do not. Empirical investigations with different LMs show that: (1) Our proposed method yields higher-quality instruction tuning data than Self-Instruct, (2) It improves performances of both vanilla and instructiontuned LMs by significant margins, and (3) Smaller instruction-tuned LMs generate more useful outputs than their larger un-tuned counterparts. Our codebase is available at https: //github.com/IBM/ensemble-instruct.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Making, Not Taking, the Best of N cites Training language models to follow instructions with human feedback
- QLoRA: Efficient Finetuning of Quantized LLMs cites Finetuned Language Models are Zero-Shot Learners
- QLoRA: Efficient Finetuning of Quantized LLMs cites Training language models to follow instructions with human feedback
- WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions cites Finetuned Language Models are Zero-Shot Learners
- WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions cites Training language models to follow instructions with human feedback
- Finetuned Language Models are Zero-Shot Learners cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Finetuned Language Models are Zero-Shot Learners
Cited by top-tier papers2
Builds on13
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
- Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingStella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley et al.ICML 2023 · 1,822 citations
- WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex InstructionsCan Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng et al.ICLR 2024 · 1,206 citations
Related papers
- REInstruct: Building Instruction Data from Unlabeled CorpusShu Chen, Xinyan Guan, Yaojie Lu, Hongyu Lin et al.ACL Findings 2024 · 3 citations
- SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-ReflectionLiangxin Liu, Xuebo Liu, Derek F. Wong, Dongfang Li et al.NeurIPS 2024 · 49 citations
- Review-Instruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language ModelsJiangxu Wu, Cong Wang, Tianhuang Su, Jun Yang et al.ACL Findings 2025
- Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with NothingZhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng et al.ICLR 2025
- SelfCodeAlign: Self-Alignment for Code GenerationYuxiang Wei, Federico Cassano, Jiawei Liu, Yifeng Ding et al.NeurIPS 2024 · 79 citations
