UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models
Abstract
We introduce UniCon, a novel architecture designed to enhance control and efficiency in training adapters for large-scale diffusion models. Unlike existing methods that rely on bidirectional interaction between the diffusion model and control adapter, UniCon implements a unidirectional flow from the diffusion network to the adapter, allowing the adapter alone to generate the final output. UniCon reduces computational demands by eliminating the need for the diffusion model to compute and store gradients during adapter training. Our results indicate that UniCon reduces GPU memory usage by one-third and increases training speed by 2.3 times, while maintaining the same adapter parameter size. Additionally, without requiring extra computational resources, UniCon enables the training of adapters with double the parameter volume of existing ControlNets. In a series of image conditional generation tasks, UniCon has demonstrated precise responsiveness to control inputs and exceptional generation capabilities.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
- IDAP++: Advancing Divergence-Aware Pruning with Joint Filter and Layer Optimization cites LoRA: Low-Rank Adaptation of Large Language Models
- IDAP++: Advancing Divergence-Aware Pruning with Joint Filter and Layer Optimization cites High-Resolution Image Synthesis with Latent Diffusion Models
- IDAP++: Advancing Divergence-Aware Pruning with Joint Filter and Layer Optimization cites Denoising Diffusion Probabilistic Models
- MatLat: Material Latent Space for PBR Texture Generation cites LoRA: Low-Rank Adaptation of Large Language Models
- Directly Denoising Diffusion Models cites High-Resolution Image Synthesis with Latent Diffusion Models
- Directly Denoising Diffusion Models cites Denoising Diffusion Probabilistic Models
- Directly Denoising Diffusion Models cites Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding cites High-Resolution Image Synthesis with Latent Diffusion Models
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding cites Denoising Diffusion Probabilistic Models
Cited by top-tier papers2
- MatLat: Material Latent Space for PBR Texture GenerationKyeongmin Yeo, Yunhong Min, Jaihoon Kim, Minhyuk SungCVPR 2026 · 2 citations
- IDAP++: Advancing Divergence-Aware Pruning with Joint Filter and Layer OptimizationAleksei Samarin, Artem A. Nazarenko, Egor Kotenko, Alexander Savelev et al.SIGMOD 2026
Builds on37
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 11,724 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
Related papers
- UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image GenerationLunhao Duan, Shanshan Zhao, Wenjun Yan, Yinglun Li et al.CVPR 2025
- DivControl: Knowledge Diversion for Controllable Image GenerationYucheng Xie, Fu Feng, Ruixiao Shi, Jing Wang et al.AAAI 2026 · 4 citations
- CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image GenerationKangfu Mei, Mauricio Delbracio, Hossein Talebi, Zhengzhong Tu et al.CVPR 2024 · 11 citations
- Reviving ConvNeXt for Efficient Convolutional Diffusion ModelsTaesung Kwon, Lorenzo Bianchi, Lennart Wittke, Felix Watine et al.CVPR 2026 · 1 citation
- Universal Few-shot Spatial Control for Diffusion ModelsKiet T. Nguyen, Chanhyuk Lee, Donggyun Kim, Dong Hoon Lee et al.NeurIPS 2025 · 2 citations
