USENIX Security2025Top-tier venue
From Purity to Peril: Backdooring Merged Models From "Harmless" Benign Components
Lijin Wang, Jingjing Wang, Tianshuo Cong, Xinlei He, Zhan Qin, Xinyi Huang
Abstract
The expansion of capabilities in large-scale models often incurs prohibitively high training costs. Fortunately, recent advancements in model merging techniques have made it possible to efficiently combine multiple large models, each designed for a specific task, into a single multi-functional model with negligible cost. Despite these advantages, there is a notable research gap regarding the security implications of model merging, particularly concerning backdoor vulnerabilities. In this study, we introduce a novel supply chain threat under the model merging scenario: multiple ostensibly benign models can be merged into a backdoored model. To rigorously explore this threat, we propose MergeBackdoor, a versatile training framework designed to suppress backdoor behaviors in upstream models before merging, while simultaneously ensuring the emergence of the backdoor when these models are merged. Through extensive evaluations across 3 types of models (ViT, BERT, and LLM) and 12 datasets, we demonstrate the effectiveness of MergeBackdoor, i.e., the attack success rates (ASRs) of the upstream models before merging are all at a random-guessing level, and the ASRs can reach nearly 1.0 for the final merged model. Besides conducting an in-depth analysis of MergeBackdoor's underlying mechanism, we further demonstrate that even the most knowledgeable detectors fail to identify the anomalies in these models before merging. We highlight that our findings underscore the critical need for security audit throughout the entire merging pipeline.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- TIES-Merging: Resolving Interference When Merging Models cites Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
- TIES-Merging: Resolving Interference When Merging Models cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- TIES-Merging: Resolving Interference When Merging Models cites Linear Mode Connectivity and the Lottery Ticket Hypothesis
- TIES-Merging: Resolving Interference When Merging Models cites Learning Transferable Visual Models From Natural Language Supervision
- BadTV: Unveiling Backdoor Threats in Third-Party Task Vectors cites TIES-Merging: Resolving Interference When Merging Models
- BadTV: Unveiling Backdoor Threats in Third-Party Task Vectors cites Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
- BadTV: Unveiling Backdoor Threats in Third-Party Task Vectors cites Linear Mode Connectivity and the Lottery Ticket Hypothesis
- BadTV: Unveiling Backdoor Threats in Third-Party Task Vectors cites Learning Transferable Visual Models From Natural Language Supervision
- PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time cites Linear Mode Connectivity and the Lottery Ticket Hypothesis
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time cites Learning Transferable Visual Models From Natural Language Supervision
- INDEXGUARD: Index-only Backdoor Vetting for Secure Federated PEFT of Large Language Models cites PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning
- Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging cites TIES-Merging: Resolving Interference When Merging Models
- From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging cites Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
- From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers6
- An Empirical Study of Observability Limits in Advanced Software Supply Chain AttacksZhuoran Tan, Wenbo Guo, Jiewen Luo, Taylor Brierley et al.CCS 2026 · 3 citations
- BadTV: Unveiling Backdoor Threats in Third-Party Task VectorsChia-Yi Hsu, Yu-Lin Tsai, Zhe Yu, Yan-Lun Chen et al.CCS 2026 · 2 citations
- From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model MergingZhenqian Zhu, Yamin Hu, Yiya Diao, Weixiang Li et al.ICML 2026 · 1 citation
- PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-TuningZhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin et al.S&P 2025
- INDEXGUARD: Index-only Backdoor Vetting for Secure Federated PEFT of Large Language ModelsJavad Dogani, Devriş İşler, Nikolaos LaoutarisICML 2026
Builds on31
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timeMitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs et al.ICML 2022 · 1,464 citations
- TIES-Merging: Resolving Interference When Merging ModelsPrateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel et al.NeurIPS 2023 · 999 citations
- Linear Mode Connectivity and the Lottery Ticket HypothesisJonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, Michael CarbinICML 2020 · 750 citations
Related papers
- Merge Hijacking: Backdoor Attacks to Model Merging of Large Language ModelsZenghui Yuan, Yangming Xu, Jiawen Shi, Pan Zhou et al.ACL 2025 · 5 citations
- Here's a Free Lunch: Sanitizing Backdoored Models with Model MergeAnsh Arora, Xuanli He, Maximilian Mozes, Srinibas Swain et al.ACL Findings 2024 · 9 citations
- BadMerging: Backdoor Attacks Against Model MergingJinghuai Zhang, Jianfeng Chi, Zheng Li, Kunlin Cai et al.CCS 2024 · 5 citations
- Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware SubspaceJinluan Yang, Anke Tang, Didi Zhu, Zhengyu Chen et al.ICLR 2025
- Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language ModelsSan Kim, Gary LeeACL 2026
