Unified Personalized Understanding, Generating and Editing
Yu Zhong, Tianwei Lin, Ruike Zhu, Yuqian Yuan, Haoyu Zheng, Liang Liang, Wenqiao Zhang, Feifei Shao, Haoyuan Li, Wanggui He, Hao Jiang, Yueting Zhuang
摘要
Unified large multimodal models (LMMs) have achieved remarkable progress in general-purpose multimodal understanding and generation. However, they still operate under a one-size-fits-all''paradigm and struggle to model user-specific concepts (e.g., generate a photo of ) in a consistent and controllable manner. Existing personalization methods typically rely on external retrieval, which is inefficient and poorly integrated into unified multimodal pipelines. Recent personalized unified models introduce learnable soft prompts to encode concept information, yet they either couple understanding and generation or depend on complex multi-stage training, leading to cross-task interference and ultimately to fuzzy or misaligned personalized knowledge. We present OmniPersona, an end-to-end personalization framework for unified LMMs that, for the first time, integrates personalized understanding, generation, and image editing within a single architecture. OmniPersona introduces structurally decoupled concept tokens, allocating dedicated subspaces for different tasks to minimize interference, and incorporates an explicit knowledge replay mechanism that propagates personalized attribute knowledge across tasks, enabling consistent personalized behavior. To systematically evaluate unified personalization, we propose OmniPBench, extending the public UnifyBench concept set with personalized editing tasks and cross-task evaluation protocols integrating understanding, generation, and editing. Experimental results demonstrate that OmniPersona delivers competitive and robust performance across diverse personalization tasks. We hope OmniPersona will serve as a strong baseline and spur further research on controllable, unified personalization.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow MatchingOnkar Susladkar, Tushar Prakash, Gayatri Deshmukh, Kiet Nguyen 等ICML 2026
- Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography AnalysisTianwei Lin, Zhongwei Qiu, Jie Cao, Jiang Liu 等ICML 2026
它引用的顶会 Paper22
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual InversionRinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik 等ICLR 2023 · 被引用 464 次
- Show-o2: Improved Native Unified Multimodal ModelsJinheng Xie, Zhenheng Yang, Mike Zheng ShouNeurIPS 2025 · 被引用 261 次
- UniTok: a Unified Tokenizer for Visual Generation and UnderstandingChuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang 等NeurIPS 2025 · 被引用 164 次
- BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active AnnotationWenqiao Zhang, Lei Zhu, James Hallinan, Shengyu Zhang 等CVPR 2022 · 被引用 115 次
- InstantBooth: Personalized Text-to-Image Generation without Test-Time FinetuningJing Shi, Wei Xiong, Zhe Lin, Hyun Joon JungCVPR 2024 · 被引用 115 次
相关 Paper
- UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept TokensRuichuan An, Sihan Yang, Renrui Zhang, Zijun Shen 等NeurIPS 2025 · 被引用 61 次
- TUNA: Taming Unified Visual Representations for Native Unified Multimodal ModelsZhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou 等CVPR 2026 · 被引用 36 次
- Omni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept PersonalizationTsai-Shien Chen, Aliaksandr Siarohin, Gordon Guocheng Qian, Kuan-Chieh Jackson Wang 等CVPR 2026 · 被引用 4 次
- Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and BeyondTianxin Wei, Bowen Jin, Ruirui Li, Hansi Zeng 等ICLR 2024 · 被引用 46 次
- MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal ModelsMingrui Wu, Hang Liu, Jiayi Ji, Xiaoshuai Sun 等CVPR 2026 · 被引用 5 次
