Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMs
Ziyi Zhao, Chongming Gao, Yang Zhang, Haoyan Liu, Weinan Gan, Huifeng Guo, Yong Liu, Fuli Feng
Abstract
Personalization in Large Language Models (LLMs) often relies on user-specific soft prompts. However, these prompts become obsolete when the foundation model is upgraded, necessitating costly, full-scale retraining. To overcome this limitation, we propose the Prompt-level User Migration Adapter (PUMA), a lightweight framework to efficiently migrate personalized prompts across incompatible models. PUMA utilizes a parameter-efficient adapter to bridge the semantic gap, combined with a group-based user selection strategy to significantly reduce training costs. Experiments on three large-scale datasets show our method matches or even surpasses the performance of retraining from scratch, reducing computational cost by up to 98%. The framework demonstrates strong generalization across diverse model architectures and robustness in advanced scenarios like chained and aggregated migrations, offering a practical path for the sustainable evolution of personalized AI by decoupling user assets from the underlying models.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext f989655a-19b2-4a84-8b17-5bebf782917cCited by top-tier papers5
- NextQuill: Causal Preference Modeling for Enhancing LLM PersonalizationXiaoyan Zhao, Juntao You, Yang Zhang, Wenjie Wang et al.ICLR 2026 · 38 citations
- Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form GenerationChengbing Wang, Yang Zhang, Wenjie Wang, Xiaoyan Zhao et al.ICLR 2026 · 35 citations
- VINCIE: Unlocking In-context Image Editing from VideoLeigang Qu, Feng Cheng, Ziyan Yang, Qi Zhao et al.ICLR 2026 · 18 citations
- TTOM: Test-Time Optimization and Memorization for Compositional Video GenerationLeigang Qu, Ziyang Wang, Na Zheng, Wenjie Wang et al.ICLR 2026 · 6 citations
- SASFT: Sparse Autoencoder-guided Supervised Finetuning to Mitigate Unexpected Code-Switching in LLMsBoyi Deng, Yu Wan, Baosong Yang, Fei Huang et al.ICLR 2026 · 2 citations
Builds on14
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Beyond neural scaling laws: beating power law scaling via data pruningBen Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli et al.NeurIPS 2022 · 720 citations
- Coresets for Data-efficient Training of Machine Learning ModelsBaharan Mirzasoleiman, Jeff A. Bilmes, Jure LeskovecICML 2020 · 494 citations
- Selection via Proxy: Efficient Data Selection for Deep LearningCody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman et al.ICLR 2020 · 462 citations
Related papers
- PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model PatchesRana Muhammad Shahroz, Pingzhi Li, Sukwon Yun, Zhenyu Wang et al.ICLR 2025
- Instant Personalized Large Language Model Adaptation via HypernetworkZhaoxuan Tan, Zixuan Zhang, Haoyang Wen, Zheng Li et al.ACL 2026 · 7 citations
- Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMsZhaowei Zhang, Fengshuo Bai, Qizhi Chen, Chengdong Ma et al.ICLR 2025
- Efficient and Privacy-Preserving Soft Prompt Transfer for LLMsXun Wang, Jing Xu, Franziska Boenisch, Michael Backes et al.ICML 2025
- CAR-LoRA: Training Compression-Aware and Robust LoRA Adapters for Evolving LLMsRana Muhammad Shahroz Khan, Zhen Tan, Ruichen Zhang, Hua Wei et al.ICLR 2026
