Lune

ACL Findings2024

LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement

Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipalli, Michael W. Mahoney, Kurt Keutzer, Amir Gholami

Abstract

Pretrained large language models (LLMs) are currently state-of-the-art for solving the vast majority of natural language processing tasks. While many real-world applications still require fine-tuning to reach satisfactory levels of performance, many of them are in the lowdata regime, making fine-tuning challenging. To address this, we propose LLM2LLM, a targeted and iterative data augmentation strategy that uses a teacher LLM to enhance a small seed dataset by augmenting additional data that can be used for fine-tuning on a specific task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data, (2) evaluates and extracts data points that the model gets wrong, and (3) uses a teacher LLM to generate synthetic data based on these incorrect data points, which are then added back into the training data. This approach amplifies the signal from incorrectly predicted data points by the LLM during training and reintegrates them into the dataset to focus on more challenging examples for the LLM. Our results show that LLM2LLM significantly enhances the performance of LLMs in the low-data regime, outperforming both traditional fine-tuning and other data augmentation baselines. LLM2LLM reduces the dependence on labor-intensive data curation and paves the way for more scalable and performant LLM solutions, allowing us to tackle data-constrained domains and tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular fine-tuning in the low-data regime using a Llama-2-7B student model. Our code is available at https: //github.com/SqueezeAILab/LLM2LLM.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

28 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Instruction Pre-Training: Language Models are Supervised Multitask Learners cites Measuring Massive Multitask Language Understanding
  • Instruction Pre-Training: Language Models are Supervised Multitask Learners cites Finetuned Language Models are Zero-Shot Learners
  • SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning cites Measuring Massive Multitask Language Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers28

  1. Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMsXuan Zhang, Chao Du, Tianyu Pang, Qian Liu et al.NeurIPS 2024 · 177 citations
  2. SiriuS: Self-improving Multi-agent Systems via Bootstrapped ReasoningWanjia Zhao, Mert Yüksekgönül, Shirley Wu, James Y. ZouNeurIPS 2025 · 49 citations
  3. ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language ModelsYuzhe Gu, Ziwei Ji, Wenwei Zhang, Chengqi Lyu et al.NeurIPS 2024 · 20 citations
  4. Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language ModelsFangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu et al.ACL 2025 · 14 citations
  5. Instruction Pre-Training: Language Models are Supervised Multitask LearnersDaixuan Cheng, Yuxian Gu, Shaohan Huang, Junyu Bi et al.EMNLP 2024 · 13 citations

Builds on31

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  4. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  5. Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations

Related papers

  1. Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text AugmentationJán Cegin, Branislav Pecher, Jakub Simko, Ivan Srba et al.EMNLP Findings 2025
  2. Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimesNabeel Seedat, Nicolas Huynh, Boris van Breugel, Mihaela van der SchaarICML 2024 · 61 citations
  3. iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool UseYirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu et al.EMNLP 2025
  4. Boosting Zero-Shot Crosslingual Performance using LLM-Based Augmentations with Effective Data SelectionBarah Fazili, Ashish Sunil Agrawal, Preethi JyothiACL Findings 2024 · 4 citations
  5. Task Oriented In-Domain Data AugmentationXiao Liang, Xinyu Hu, Simiao Zuo, Yeyun Gong et al.EMNLP 2024 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines