ProFuser: Progressive Fusion of Large Language Models
Tianyuan Shi, Fanqi Wan, Canbin Huang, Xiaojun Quan, Chenliang Li, Ming Yan, Ji Zhang, Minhua Huang, Wu Kai
摘要
While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during training. Existing fusion methods primarily focus on the training mode that uses cross entropy on ground truth in a teacher-forcing setup to measure a model's advantage, which may provide limited insight towards model advantage. In this paper, we introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes. Our method evaluates model advantage not only through cross entropy during training but also by considering inference outputs, providing a more comprehensive assessment. To combine the two modes effectively, we introduce ProFuser to progressively transition from inference mode to training mode. To validate ProFuser's effectiveness, we fused three models, including Vicuna-7B-v1.5, Llama-2-7B-Chat, and MPT-7B-8K-Chat, and demonstrated the improved performance in knowledge, reasoning, and safety compared to baseline methods.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
已有 3 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。
脉络
- InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion 引用了 TinyBERT: Distilling BERT for Natural Language Understanding
- InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion 引用了 Measuring Massive Multitask Language Understanding
- TruthfulQA: Measuring How Models Mimic Human Falsehoods 引用了 Measuring Massive Multitask Language Understanding
- Weighted-Reward Preference Optimization for Implicit Model Fusion 引用了 Measuring Massive Multitask Language Understanding
- Weighted-Reward Preference Optimization for Implicit Model Fusion 引用了 TruthfulQA: Measuring How Models Mimic Human Falsehoods
引用它的顶会 Paper3
- InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model FusionYuanyi Wang, Zhaoyi Yan, Yiming Zhang, Qi Zhou 等NeurIPS 2025 · 12 次引用
- Omni-DNA: A Genomic Model Supporting Sequence Understanding, Long-context, and Textual AnnotationZehui Li, Vallijah Subasri, Yifei Shen, Dongsheng Li 等NeurIPS 2025 · 6 次引用
- Weighted-Reward Preference Optimization for Implicit Model FusionZiyi Yang, Fanqi Wan, Longguang Zhong, Tianyuan Shi 等ICLR 2025
它引用的 Paper15
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 18,833 次引用
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou 等ICLR 2021 · 7,905 次引用
- TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 次引用
- WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 次引用
- TinyBERT: Distilling BERT for Natural Language UnderstandingXiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang 等EMNLP Findings 2020 · 1,787 次引用
相关 Paper
- Knowledge Fusion of Large Language ModelsFanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan 等ICLR 2024 · 113 次引用
- InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language ModelsYanggan Gu, Yuanyi Wang, Zhaoyi Yan, Yiming Zhang 等NeurIPS 2025 · 9 次引用
- AdaFuse: Adaptive Ensemble Decoding for Large Language ModelsChengming Cui, Tianxin Wei, Ziyi Chen, Ruizhong Qiu 等ACL 2026
- MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction FusionQizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li 等ACL 2025 · 27 次引用
- Fusing Highly Specialized Language Models for Comprehensive ExpertiseNing Ding, Yulin Chen, Ganqu Cui, Xingtai Lv 等ACL 2025
