AlpaGasus: Training a Better Alpaca with Fewer Data

Large language models (LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data. However, widely used IFT datasets (e.g., ALPACA's 52k data) surprisingly contain many lowquality instances with incorrect or irrelevant responses, which are misleading and detrimental to IFT. In this paper, we propose a simple and effective data selection strategy that automatically identifies and filters out low-quality data using a strong LLM (e.g., ChatGPT). To this end, we introduce ALPAGASUS, which is finetuned on only 9k high-quality data filtered from the 52k ALPACA data. ALPAGASUS significantly outperforms the original ALPACA as evaluated by GPT-4 on multiple test sets and the controlled human evaluation. Its 13B variant matches > 90% performance of its teacher LLM (i.e., Text-Davinci-003 generating the 52k data) on test tasks. It also provides 5.7x faster training, reducing the training time for a 7B variant from 80 minutes (for ALPACA) to 14 minutes 1 . Moreover, the experiments prove the efficacy of our method across diverse datasets, base models, and LLM filters. Overall, ALPAGASUS demonstrates a novel data-centric IFT paradigm that can be generally applied to instruction-tuning data, leading to faster training and better instruction-following models. Our project page is available at: https://lichang-chen.github.io/AlpaGasus/ . * Equal Contribution. This work was done when Lichang Chen and Jun Yan interned at Samsung Research America. 1 We apply IFT for the same number of epochs as ALPACA(7B) but on fewer data, using 4×NVIDIA A100 (80GB) GPUs and following the original ALPACA setting and hyperparameters.