LoFA: Learning to Predict Personalized Prior for Fast Adaptation of Visual Generative Models
Yiming Hao, Mutian Xu, Chongjie Ye, Jie Qin, Shunlin Lu, Yipeng Qin, Xiaoguang Han
2026Year
Abstract
a two-stage hypernetwork: first predicting relative distribution patterns that capture key adaptation regions, then using these to guide final LoRA weight prediction. Extensive experiments demonstrate that our method consistently predicts high-quality personalized priors within seconds, across multiple tasks and user prompts, even outperforming conventional LoRA that requires hours of processing. Project page: jaeger416.github.io/lofa.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on34
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
Related papers
- HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait SynthesisMengtian Li, Jinshu Chen, Wanquan Feng, Bingchuan Li et al.CVPR 2025
- Text-to-LoRA: Instant Transformer AdaptionRujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko LangeICML 2025
- LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image GenerationDonald Shenaj, Ondrej Bohdal, Mete Ozay, Pietro Zanuttigh et al.ICCV 2025
- Instant Personalized Large Language Model Adaptation via HypernetworkZhaoxuan Tan, Zixuan Zhang, Haoyang Wen, Zheng Li et al.ACL 2026 · 7 citations
- LoRAGen: Structure-Aware Weight Space Learning for LoRA GenerationHao Huang, Jingtao Ding, Mengqi Liao, Xin Wang et al.ICLR 2026
