EMNLP Findings2024
Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models
Jie Chen, Yupeng Zhang, Bingning Wang, Xin Zhao, Ji-Rong Wen, Weipeng Chen
Abstract
Synthetic data has been proposed as a solution to address the issue of high-quality data scarcity in the training of large language models (LLMs). Studies have shown that synthetic data can effectively improve the performance of LLMs on downstream benchmarks. However, despite its potential benefits, our analysis suggests that there may be inherent flaws in synthetic data. The uniform format of synthetic data can lead to pattern overfitting and cause significant shifts in the output distribution, thereby reducing the model's instructionfollowing capabilities. Our work delves into these specific flaws associated with questionanswer (Q-A) pairs, a prevalent type of synthetic data, and presents a method based on unlearning techniques to mitigate these flaws. The empirical results demonstrate the effectiveness of our approach, which can reverse the instruction-following issues caused by pattern overfitting without compromising performance on benchmarks at relatively low cost. Our work has yielded key insights into the effective use of synthetic data, aiming to promote more robust and efficient LLM training.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Measuring Massive Multitask Language Understanding cites Language Models are Few-Shot Learners
- Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning cites Measuring Massive Multitask Language Understanding
- Detecting Pretraining Data from Large Language Models cites Language Models are Few-Shot Learners
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models cites Language Models are Few-Shot Learners
Cited by top-tier papers4
- LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMsPei-Fu Guo, Yun-Da Tsai, Chun-Chia Hsu, Kai-Xin Chen et al.ACL 2026 · 1 citation
- Montessori-Instruct: Generate Influential Training Data Tailored for Student LearningXiaochuan Li, Zichun Yu, Chenyan XiongICLR 2025
- From Selection to Refinement: Iterative Optimization for Instruction DataHang Hu, Ziyan Liu, Rujie Wen, Ruihui Hou et al.ACL 2026
- Understanding Synthetic Context Extension via Retrieval HeadsXinyu Zhao, Fangcong Yin, Greg DurrettICML 2025
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language ModelsLonghui Yu, Weisen Jiang, Han Shi, Jincheng Yu et al.ICLR 2024 · 637 citations
- Large Language Model UnlearningYuanshun Yao, Xiaojun Xu, Yang LiuNeurIPS 2024 · 365 citations
- Detecting Pretraining Data from Large Language ModelsWeijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang et al.ICLR 2024 · 365 citations
Related papers
- MAIN: Mutual Alignment Is Necessary for instruction tuningFanyi Yang, Jianfeng Liu, Xin Zhang, Haoyu Liu et al.EMNLP 2025
- On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A SurveyLin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao et al.ACL Findings 2024 · 131 citations
- Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text ClassificationHsun-Yu Kuo, Yin-Hsiang Liao, Yu-Chieh Chao, Wei-Yun Ma et al.ICLR 2025
- Forewarned is Forearmed: Harnessing LLMs for Data Synthesis via Failure-induced ExplorationQintong Li, Jiahui Gao, Sheng Wang, Renjie Pi et al.ICLR 2025
- Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and PitfallsFeiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad et al.EMNLP 2025
