IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts
Bohan Zeng, Shanglin Li, Yutang Feng, Ling Yang, Juan Zhang, Hong Li, Jiaming Liu, Conghui He, Wentao Zhang, Jianzhuang Liu, Baochang Zhang, Shuicheng Yan
Abstract
Recent advances in 3D generation have been remarkable, with methods such as DreamFusion leveraging large-scale text-to-image diffusion-based models to guide 3D object generation. These methods enable the synthesis of detailed and photorealistic textured objects. However, the appearance of 3D objects produced by such text-to-3D models is often unpredictable, and it is hard for single-imageto-3D methods to deal with images lacking a clear subject, complicating the generation of appearance-controllable 3D objects from complex images. To address these challenges, we present IPDreamer, a novel method that captures intricate appearance features from complex Image Prompts and aligns the synthesized 3D object with these extracted features, enabling high-fidelity, appearance-controllable 3D object generation. Our experiments demonstrate that IPDreamer consistently generates high-quality 3D objects that align with both the textual and complex image prompts, highlighting its promising capability in appearance-controlled, complex 3D object generation.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
8 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
- Segment Anything cites Language Models are Few-Shot Learners
- Segment Anything cites Learning Transferable Visual Models From Natural Language Supervision
- Any2anytryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks cites LoRA: Low-Rank Adaptation of Large Language Models
- Any2anytryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks cites Learning Transferable Visual Models From Natural Language Supervision
- Any2anytryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks cites SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation
- SketchDream: Sketch-based Text-To-3D Generation and Editing cites Learning Transferable Visual Models From Natural Language Supervision
- Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
- SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation cites LoRA: Low-Rank Adaptation of Large Language Models
- SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation cites Learning Transferable Visual Models From Natural Language Supervision
- SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation cites Denoising Diffusion Probabilistic Models
Cited by top-tier papers8
- UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept TokensRuichuan An, Sihan Yang, Renrui Zhang, Zijun Shen et al.NeurIPS 2025 · 61 citations
- SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven GenerationYuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang et al.CVPR 2024 · 34 citations
- SketchDream: Sketch-based Text-To-3D Generation and EditingFeng-Lin Liu, Hongbo Fu, Yu-Kun Lai, Lin GaoSIGGRAPH 2024 · 30 citations
- Any2anytryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing TasksHailong Guo, Bohan Zeng, Yiren Song, Wentao Zhang et al.ICCV 2025 · 13 citations
- MVD^2: Efficient Multiview 3D Reconstruction for Multiview DiffusionXin-Yang Zheng, Hao Pan, Yu-Xiao Guo, Xin Tong et al.SIGGRAPH 2024 · 12 citations
Builds on63
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
Related papers
- DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D DataQihao Liu, Yi Zhang, Song Bai, Adam Kortylewski et al.CVPR 2024 · 4 citations
- DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion ModelsYukang Cao, Yan-Pei Cao, Kai Han, Ying Shan et al.CVPR 2024
- DreamBooth3D: Subject-Driven Text-to-3D GenerationAmit Raj, Srinivas Kaza, Ben Poole, Michael Niemeyer et al.ICCV 2023 · 280 citations
- DreamHuman: Animatable 3D Avatars from TextNikos Kolotouros, Thiemo Alldieck, Andrei Zanfir, Eduard Gabriel Bazavan et al.NeurIPS 2023 · 136 citations
- Generating compositional scenes via Text-to-image RGBA Instance GenerationAlessandro Fontanella, Petru-Daniel Tudosiu, Yongxin Yang, Shifeng Zhang et al.NeurIPS 2024 · 13 citations
