Think-Then-Generate: Structural Chain-of-Thought Reasoning for Consistent 3D Generation
Xinyue Liu, Jin Liu, Hongbo Wang, Ran He, Huaibo Huang
摘要
Recently, generating 3D assets using visual priors from pretrained diffusion models has shown remarkable results. However, due to the inherent lack of 3D geometric priors in 2D diffusion, the synthesized results often suffer from spatial hallucination and multi-view inconsistency. To address this limitation, we propose Thoughtful3D, a novel framework that enhances 3D content generation quality by introducing structural chain-of-thought (CoT) reasoning to alleviate inconsistent issues and mitigate hallucinations. Specifically, we design a dual-phase structural CoT strategy: (1) 3DBlueprint-CoT explicitly plans the 3D generation process through textual semantic parsing and logical deduction during the initialization phase. (2) 3DRefine-CoT dynamically evaluates latent inconsistencies by analyzing multiple renderings, employing a multi-round iterative refinement mechanism to suppress hallucinations and enhance cross-view consistency. To further promote consistency across views, we propose a Cross-view Semantic Appearance Alignment strategy that enhances multi-view consistency by establishing dynamic geometric associations between the same features from different viewpoints. Extensive experiments demonstrate that Thoughtful3D significantly improves the quality and consistency of generated 3D assets.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper37
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 被引用 22,562 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 被引用 11,724 次
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 被引用 5,687 次
相关 Paper
- Hallo3D: Multi-Modal Hallucination Detection and Mitigation for Consistent 3D Content GenerationHongbo Wang, Jie Cao, Jin Liu, Xiaoqiang Zhou 等NeurIPS 2024 · 被引用 10 次
- SweetDreamer: Aligning Geometric Priors in 2D diffusion for Consistent Text-to-3DWeiyu Li, Rui Chen, Xuelin Chen, Ping TanICLR 2024 · 被引用 155 次
- Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion PriorsYukang Lin, Haonan Han, Chaoqun Gong, Zunnan Xu 等ACM MM 2024 · 被引用 20 次
- Retrieval-Augmented Score Distillation for Text-to-3D GenerationJunyoung Seo, Susung Hong, Wooseok Jang, Inès Hyeonsu Kim 等ICML 2024 · 被引用 14 次
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D PriorCheng Chen, Xiaofeng Yang, Fan Yang, Chengzeng Feng 等CVPR 2024
