ProFuser: Progressive Fusion of Large Language Models
Tianyuan Shi, Fanqi Wan, Canbin Huang, Xiaojun Quan, Chenliang Li, Ming Yan, Ji Zhang, Minhua Huang, Wu Kai
Abstract
While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during training. Existing fusion methods primarily focus on the training mode that uses cross entropy on ground truth in a teacher-forcing setup to measure a model's advantage, which may provide limited insight towards model advantage. In this paper, we introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes. Our method evaluates model advantage not only through cross entropy during training but also by considering inference outputs, providing a more comprehensive assessment. To combine the two modes effectively, we introduce ProFuser to progressively transition from inference mode to training mode. To validate ProFuser's effectiveness, we fused three models, including Vicuna-7B-v1.5, Llama-2-7B-Chat, and MPT-7B-8K-Chat, and demonstrated the improved performance in knowledge, reasoning, and safety compared to baseline methods.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion cites TinyBERT: Distilling BERT for Natural Language Understanding
- InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion cites Measuring Massive Multitask Language Understanding
- TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
- Weighted-Reward Preference Optimization for Implicit Model Fusion cites Measuring Massive Multitask Language Understanding
- Weighted-Reward Preference Optimization for Implicit Model Fusion cites TruthfulQA: Measuring How Models Mimic Human Falsehoods
Cited by top-tier papers3
- InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model FusionYuanyi Wang, Zhaoyi Yan, Yiming Zhang, Qi Zhou et al.NeurIPS 2025 · 12 citations
- Omni-DNA: A Genomic Model Supporting Sequence Understanding, Long-context, and Textual AnnotationZehui Li, Vallijah Subasri, Yifei Shen, Dongsheng Li et al.NeurIPS 2025 · 6 citations
- Weighted-Reward Preference Optimization for Implicit Model FusionZiyi Yang, Fanqi Wan, Longguang Zhong, Tianyuan Shi et al.ICLR 2025
Builds on15
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
- WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
- TinyBERT: Distilling BERT for Natural Language UnderstandingXiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang et al.EMNLP Findings 2020 · 1,787 citations
Related papers
- Knowledge Fusion of Large Language ModelsFanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan et al.ICLR 2024 · 113 citations
- InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language ModelsYanggan Gu, Yuanyi Wang, Zhaoyi Yan, Yiming Zhang et al.NeurIPS 2025 · 9 citations
- AdaFuse: Adaptive Ensemble Decoding for Large Language ModelsChengming Cui, Tianxin Wei, Ziyi Chen, Ruizhong Qiu et al.ACL 2026
- MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction FusionQizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li et al.ACL 2025 · 27 citations
- Fusing Highly Specialized Language Models for Comprehensive ExpertiseNing Ding, Yulin Chen, Ganqu Cui, Xingtai Lv et al.ACL 2025
