Lune

ACL Findings2026

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections

Haebin Shin, Lei Ji, Xiao Liu, Zhiwei Yu, Hyunwoo Yoo, Qi Chen, Yeyun Gong

Abstract

As numerous instruction-tuning datasets continue to emerge, dynamically balancing and optimizing their mixtures has become a critical challenge. To address this, we propose DY-NAMIXSFT, a dynamic and automated method for instruction-tuning dataset mixture optimization. We formulate the problem as a multiarmed bandit setup and introduce a Priorscaled Boltzmann Exploration that softly anchors the updated sampling distribution to the original dataset proportions, thereby preserving the inherent diversity and coverage of the collection. Sampling probabilities are updated using a lightweight 1-Step Look-ahead Reward, reflecting how much the dataset contributes to improving the model's performance at its current state. We demonstrate that DYNAMIXSFT effectively optimizes the TÜLU-2-mixture and TÜLU-3-mixture collections across 10 benchmarks, while introducing minimal computational overhead over naive sampling. Furthermore, we provide a comprehensive analysis and visualizations to offer deeper insights into the adaptive dynamics of our method.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions cites Measuring Massive Multitask Language Understanding
  • WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions cites Finetuned Language Models are Zero-Shot Learners
  • TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on23

  1. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  2. Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
  4. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code GenerationJiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming ZhangNeurIPS 2023 · 2,317 citations
  5. WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex InstructionsCan Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng et al.ICLR 2024 · 1,206 citations

Related papers

  1. MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic SpaceYicheng Chen, Yining Li, Kai Hu, Zerun Ma et al.ACL Findings 2025 · 2 citations
  2. SMART: Submodular Data Mixture Strategy for Instruction TuningH. S. V. N. S. Kowndinya Renduchintala, Sumit Bhatia, Ganesh RamakrishnanACL Findings 2024 · 2 citations
  3. Data Diversity Matters for Robust Instruction TuningAlexander Bukharin, Shiyang Li, Zhengyang Wang, Jingfeng Yang et al.EMNLP Findings 2024 · 13 citations
  4. SFTMix: Elevating Language Model Instruction Tuning with Mixup RecipeYuxin Xiao, Shujian Zhang, Marzyeh Ghassemi, Wenxuan ZhouACL 2026 · 3 citations
  5. Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task AdaptationChen Dun, Mirian del Carmen Hipolito Garcia, Guoqing Zheng, Ahmed Hassan Awadallah et al.AAAI 2025 · 7 citations
Dusk over the sea between two cliffs drawn in fine vertical lines