ShapeWords: Guiding Text-to-Image Synthesis with 3D Shape-Aware Prompts
Dmitry Petrov, Pradyumn Goyal, Divyansh Shivashok, Yuanming Tao, Melinos Averkiou, Evangelos Kalogerakis
Abstract
4 TU Crete Figure 1 . ShapeWords enables 3D shape-aware text-to-image generation via mapping of shape geometries into CLIP space. Given an input 3D shape and text prompts describing desired appearance and context, our method generates images that maintain both shape fidelity and text compliance. Unlike existing methods that use view-dependent guidance like depth maps, ShapeWords generalizes to compositional settings (top row) and allows for exploration of target geometries with stylistic deviations aligned with the prompt (bottom row).
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 0b406223-015e-491a-8fcf-92edfd4d84c8Builds on31
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
Related papers
- Towards Implicit Text-Guided 3D Shape GenerationZhengzhe Liu, Yi Wang, Xiaojuan Qi, Chi-Wing FuCVPR 2022 · 59 citations
- CLIP-Forge: Towards Zero-Shot Text-to-Shape GenerationAditya Sanghi, Hang Chu, Joseph G. Lambourne, Ye Wang et al.CVPR 2022 · 206 citations
- SketchVideo: Sketch-based Video Generation and EditingFeng-Lin Liu, Hongbo Fu, Xintao Wang, Weicai Ye et al.CVPR 2025
- Shape-Aware Text-Driven Layered Video EditingYao-Chih Lee, Ji-Ze Genevieve Jang, Yi-Ting Chen, Elizabeth Qiu et al.CVPR 2023
- Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent RepresentationZibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng et al.NeurIPS 2023 · 279 citations
