Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization
Connor Dunlop, Matthew Zheng, Kavana Venkatesh, Pinar Yanardag
摘要
Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model's editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper3
- Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image GenerationZihao Wang, Yuxiang Wei, Xinpeng Zhou, Tianyu Zhang 等CVPR 2026 · 被引用 1 次
- Offline Preference Optimization for Rectified Flow with Noise-Tracked PairsYunhong Lu, Qichao Wang, Hengyuan Cao, Xiaoyin Xu 等ICML 2026 · 被引用 1 次
- Test-Time Reinforcement Learning for Flow MatchingJili Chen, Changqin Huang, Qionghao Huang, Yaxin Tu 等ICML 2026
它引用的顶会 Paper22
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 被引用 64,255 次
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 被引用 24,707 次
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
相关 Paper
- Rethinking Direct Preference Optimization in Diffusion ModelsJunyong Kang, Seohyun Lim, Kyungjune Baek, Hyunjung ShimAAAI 2026
- DSPO: Direct Score Preference Optimization for Diffusion Model AlignmentHuaisheng Zhu, Teng Xiao, Vasant G. HonavarICLR 2025
- Towards Better Optimization For Listwise Preference in Diffusion ModelsJiamu Bai, Xin Yu, Meilong Xu, Weitao Lu 等ICLR 2026 · 被引用 8 次
- Self-Supervised Direct Preference Optimization for Text-to-Image Diffusion ModelsLiang Peng, Boxi Wu, Haoran Cheng, Yibo Zhao 等NeurIPS 2025 · 被引用 2 次
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human PreferencesYunhong Lu, Qichao Wang, Hengyuan Cao, Xiaoyin Xu 等ICML 2025
