Exposing Functional Fusion: A New Class of Strategic Backdoor in Dynamic Prompt Architectures
Zeyao Liu, Zhendong Zhao, Xiaojun Chen, Xin Zhao, Yuexin Xuan, XIAOSHUANG JI
Abstract
Existing ViT backdoor attacks based on backboneoverwriting full-tuning are computationally expensive and inflict performance degradation. This has forced adversaries towards the Visual Parameter-Efficient Fine-Tuning (PEFT) paradigm, dominated by adapter-based (e.g., LoRA) and prompt-based (e.g., VPT) approaches. While adapter security has seen initial study, the risks of the burgeoning prompt-based ecosystem remain critically unexplored. We fill this critical gap, exposing how the evolution of VPT towards dynamic and context-aware architectures can facilitate a far more dangerous and emergent threat. This vulnerability arises even though these dynamic modules unlock superior benign performance. We propose VIPER, an attack framework built on a lightweight, dynamic Visual Prompt Generator (VPG) that demonstrates this vulnerability. Critically, this dynamic architecture enables Functional Fusion: an emergent phenomenon where malicious logic and benign task utility are tightly fused into the same sparse, high-magnitude parameter core. This fusion creates a formidable "hostage" dilemma, as pruning the attack necessarily destroys the benign performance. Comprehensive evaluations show VIPER effectively addresses the attacker's trilemma: VIPER not only achieves state-of-the-art performance on clean data, but also maintains near-100% ASR even under 90% VPG-module pruning (where LoRA attacks collapse), while adding only an imperceptible 0.06ms (1.16%) of inference latency. VIPER's results, driven by Functional Fusion, expose a new, paradigm-level risk in dynamic prompt architectures.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on22
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Training data-efficient image transformers & distillation through attentionHugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa et al.ICML 2021 · 8,974 citations
- MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision TransformerSachin Mehta, Mohammad RastegariICLR 2022 · 2,162 citations
- Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural NetworksBolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li et al.S&P 2019 · 1,801 citations
Related papers
- StolenLoRA: Exploring LoRA Extraction Attacks via Synthetic DataYixu Wang, Yan Teng, Yingchun Wang, Xingjun MaICCV 2025
- CVPT: Cross Visual Prompt TuningLingyun Huang, Jianxu Mao, Junfei Yi, Ziming Tao et al.ICCV 2025 · 6 citations
- RedVisor: Reasoning-Aware Prompt Injection Defense via Zero-Copy KV Cache ReuseMingrui Liu, Sixiao Zhang, Cheng Long, Kwok Yan LamICML 2026 · 2 citations
- Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt AdaptationAbhinav Jain, Swarat Chaudhuri, Thomas W. Reps, Christopher M. JermaineNeurIPS 2024 · 10 citations
- PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-TuningZhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin et al.S&P 2025
