EMNLP Findings2022
LADIS: Language Disentanglement for 3D Shape Editing
Ian Huang, Panos Achlioptas, Tianyi Zhang, Sergey Tulyakov, Minhyuk Sung, Leonidas J. Guibas
Abstract
Natural language interaction is a promising direction for democratizing 3D shape design. However, existing methods for text-driven 3D shape editing face challenges in producing decoupled, local edits to 3D shapes. We address this problem by learning disentangled latent representations that ground language in 3D geometry. To this end, we propose a complementary tool set including a novel network architecture, a disentanglement loss, and a new editing procedure. Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision. We show that our method outperforms existing SOTA methods by 20% in terms of edit locality, and up to 6.6% in terms of language reference resolution accuracy. Human evaluations additionally show that compared to the existing SOTA, our method produces shape edits that are more local, more semantically accurate, and more visually obvious. Our work suggests that by solely disentangling language representations, downstream 3D shape editing can become more local to relevant parts, even if the model was never given explicit part-based supervision.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Spice·E: Structural Priors in 3D Diffusion using Cross-Entity Attention cites Zero-Shot Text-Guided Object Generation with Dream Fields
- Spice·E: Structural Priors in 3D Diffusion using Cross-Entity Attention cites Learning Transferable Visual Models From Natural Language Supervision
- Spice·E: Structural Priors in 3D Diffusion using Cross-Entity Attention cites DreamFusion: Text-to-3D using 2D Diffusion
- StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery cites Learning Transferable Visual Models From Natural Language Supervision
- Prox-E: Fine-Grained 3D Shape Editing via Primitive-Based Abstractions cites Spice·E: Structural Priors in 3D Diffusion using Cross-Entity Attention
- Prox-E: Fine-Grained 3D Shape Editing via Primitive-Based Abstractions cites CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization
- Prox-E: Fine-Grained 3D Shape Editing via Primitive-Based Abstractions cites Blended Point Cloud Diffusion for Localized Text-Guided Shape Editing
- Prox-E: Fine-Grained 3D Shape Editing via Primitive-Based Abstractions cites DreamFusion: Text-to-3D using 2D Diffusion
- CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization cites DreamFusion: Text-to-3D using 2D Diffusion
- Zero-Shot Text-Guided Object Generation with Dream Fields cites StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery
- Zero-Shot Text-Guided Object Generation with Dream Fields cites Learning Transferable Visual Models From Natural Language Supervision
- Blended Point Cloud Diffusion for Localized Text-Guided Shape Editing cites Spice·E: Structural Priors in 3D Diffusion using Cross-Entity Attention
- Blended Point Cloud Diffusion for Localized Text-Guided Shape Editing cites CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization
- Blended Point Cloud Diffusion for Localized Text-Guided Shape Editing cites Zero-Shot Text-Guided Object Generation with Dream Fields
- Blended Point Cloud Diffusion for Localized Text-Guided Shape Editing cites Learning Transferable Visual Models From Natural Language Supervision
- Blended Point Cloud Diffusion for Localized Text-Guided Shape Editing cites DreamFusion: Text-to-3D using 2D Diffusion
- DreamFusion: Text-to-3D using 2D Diffusion cites Zero-Shot Text-Guided Object Generation with Dream Fields
- DreamFusion: Text-to-3D using 2D Diffusion cites Learning Transferable Visual Models From Natural Language Supervision
- DreamFusion: Text-to-3D using 2D Diffusion cites Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
- Parts of Speech-Grounded Subspaces in Vision-Language Models cites StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery
- Parts of Speech-Grounded Subspaces in Vision-Language Models cites Learning Transferable Visual Models From Natural Language Supervision
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers6
- Parts of Speech-Grounded Subspaces in Vision-Language ModelsJames Oldfield, Christos Tzelepis, Yannis Panagakis, Mihalis Nicolaou et al.NeurIPS 2023 · 13 citations
- Spice·E: Structural Priors in 3D Diffusion using Cross-Entity AttentionEtai Sella, Gal Fiebelman, Noam Atia, Hadar Averbuch-ElorSIGGRAPH 2024 · 7 citations
- CNS-Edit: 3D Shape Editing via Coupled Neural Shape OptimizationJingyu Hu, Ka-Hei Hui, Zhengzhe Liu, Hao Zhang et al.SIGGRAPH 2024 · 7 citations
- Blended Point Cloud Diffusion for Localized Text-Guided Shape EditingEtai Sella, Noam Atia, Ron Mokady, Hadar Averbuch-ElorICCV 2025 · 4 citations
- Prox-E: Fine-Grained 3D Shape Editing via Primitive-Based AbstractionsEtai Sella, Hao Phung, Nitay Amiel, Or Litany et al.SIGGRAPH 2026 · 2 citations
Builds on13
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
- StyleCLIP: Text-Driven Manipulation of StyleGAN ImageryOr Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or et al.ICCV 2021 · 1,437 citations
- DreamFusion: Text-to-3D using 2D DiffusionBen Poole, Ajay Jain, Jonathan T. Barron, Ben MildenhallICLR 2023 · 463 citations
- Zero-Shot Text-Guided Object Generation with Dream FieldsAjay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel et al.CVPR 2022 · 361 citations
Related papers
- ShapeTalk: A Language Dataset and Framework for 3D Shape Edits and DeformationsPanos Achlioptas, Ian Huang, Minhyuk Sung, Sergey Tulyakov et al.CVPR 2023
- InterCoser: Interactive 3D Character Creation with Disentangled Fine-Grained FeaturesYi Wang, Jian Ma, Zhuo Su, Guidong Wang et al.AAAI 2026
- EditVAE: Unsupervised Parts-Aware Controllable 3D Point Cloud Shape GenerationShidi Li, Miaomiao Liu, Christian WalderAAAI 2022 · 35 citations
- Vinedresser3D: Towards Agentic Text-guided 3D EditingYankuan Chi, Xiang Li, Zixuan Huang, James M.CVPR 2026
- RELATE3D: REfocusing Latent Adapter for Targeted local Enhancement and Editing in 3D GenerationXiao-Lei Li, Hao-Xiang Chen, Yanni Zhang, Kai Ma et al.SIGGRAPH 2025 · 2 citations
