CVPR Findings2026
Mix-to-Max: Optimizing Data Mixtures for Peak Vision-Language Efficiency
Erwei Zhao, Haijin Zeng, Weiwei Xiao, Shijie Cao, Qiben Shan, Shaocong Wu, Jingyong Su, Jie Liu
Abstract
Visual Instruction Tuning (VIT) endows Large Vision-Language Models (LVLMs) with powerful generalization capabilities across diverse visual tasks. However, the redundancy present in VIT data leads to substantial computational overhead, making data pruning critical for achieving efficient VIT. Existing pruning methods struggle to achieve a favorable balance between performance and efficiency. Sample influence-based methods rely on external validation data, causing the selected data to be biased toward samples that match the format of external data. Moreover, the computational overhead of calculating sample level influence function even exceeds the cost of training on the full dataset. To overcome these limitations, this paper casts data pruning as an optimization problem over subset mixing proportions and proposes the Mix-to-Max method. This method requires no external data and computes influence across data subsets to build an Influence Transfer Matrix (ITM). The ITM captures the directional knowledge flow, which allows for a precise estimation of synergy and redundancy among subsets to significantly improve data efficiency. Experimental results on the LLaVA-665K and Vision-Flan-186K datasets demonstrate that with just 20% of the data, Mix-to-Max achieves 98.82% and 98.45% of the full-dataset performance respectively, and reduces the total wall-clock time by over 70%. Our code is publicly available at: https://github.com/ew- zhao/Mix-to-Max.git.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
- Visual Instruction Tuning cites Flamingo: a Visual Language Model for Few-Shot Learning
- Visual Instruction Tuning cites Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
- Flamingo: a Visual Language Model for Few-Shot Learning cites Learning Transferable Visual Models From Natural Language Supervision
Builds on18
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question AnsweringPan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu et al.NeurIPS 2022 · 2,727 citations
- Deep Learning on a Data Diet: Finding Important Examples Early in TrainingMansheej Paul, Surya Ganguli, Gintare Karolina DziugaiteNeurIPS 2021 · 806 citations
Related papers
- Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment TrajectoriesNilay Naharas, Dang Nguyen, Neslihan Bulut, MohammadHossein Bateni et al.ICML 2026 · 7 citations
- Less is More: High-value Data Selection for Visual Instruction TuningZikang Liu, Kun Zhou, Wayne Xin Zhao, Dawei Gao et al.ACM MM 2025 · 3 citations
- EfficientLLaVA: Generalizable Auto-Pruning for Large Vision-language ModelsYinan Liang, Ziwei Wang, Xiuwei Xu, Jie Zhou et al.CVPR 2025
- TransPrune: Token Transition Pruning for Efficient Large Vision-Language ModelAo Li, Yuxiang Duan, Jinghui Zhang, Congbo Ma et al.CVPR 2026 · 3 citations
- Data Pruning by Information MaximizationHaoru Tan, Sitong Wu, Wei Huang, Shizhen Zhao et al.ICLR 2025
