EG4D: Explicit Generation of 4D Object without Score Distillation
Qi Sun, Zhiyang Guo, Ziyu Wan, Jing Nathan Yan, Shengming Yin, Wengang Zhou, Jing Liao, Houqiang Li
Abstract
In recent years, the increasing demand for dynamic 3D assets in design and gaming applications has given rise to powerful generative pipelines capable of synthesizing high-quality 4D objects. Previous methods generally rely on score distillation sampling (SDS) algorithm to infer the unseen views and motion of 4D objects, thus leading to unsatisfactory results with defects like over-saturation and Janus problem. Therefore, inspired by recent progress of video diffusion models, we propose to optimize a 4D representation by explicitly generating multi-view videos from one input image. However, it is far from trivial to handle practical challenges faced by such a pipeline, including dramatic temporal inconsistency, inter-frame geometry and texture diversity, and semantic defects brought by video generation results. To address these issues, we propose EG4D, a novel multi-stage framework that generates high-quality and consistent 4D assets without score distillation. Specifically, collaborative techniques and solutions are developed, including an attention injection strategy to synthesize temporal-consistent multi-view videos, a robust and efficient dynamic reconstruction method based on Gaussian Splatting, and a refinement stage with diffusion prior for semantic restoration. The qualitative results and user preference study demonstrate that our framework outperforms the baselines in generation quality by a considerable margin. Code will be released at https://github.com/jasongzy/EG4D.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
22 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
- V2M4: 4D Mesh Animation Reconstruction from a Single Monocular Video cites High-Resolution Image Synthesis with Latent Diffusion Models
- V2M4: 4D Mesh Animation Reconstruction from a Single Monocular Video cites Learning Transferable Visual Models From Natural Language Supervision
- V2M4: 4D Mesh Animation Reconstruction from a Single Monocular Video cites Denoising Diffusion Probabilistic Models
- ShapeGen4D: Towards High Quality 4D Shape Generation from Videos cites V2M4: 4D Mesh Animation Reconstruction from a Single Monocular Video
- ShapeGen4D: Towards High Quality 4D Shape Generation from Videos cites Learning Transferable Visual Models From Natural Language Supervision
- Adding Conditional Control to Text-to-Image Diffusion Models cites High-Resolution Image Synthesis with Latent Diffusion Models
- Adding Conditional Control to Text-to-Image Diffusion Models cites Learning Transferable Visual Models From Natural Language Supervision
- Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
- Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation cites High-Resolution Image Synthesis with Latent Diffusion Models
- Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation cites Learning Transferable Visual Models From Natural Language Supervision
- Puppeteer: Rig and Animate Your 3D Models cites V2M4: 4D Mesh Animation Reconstruction from a Single Monocular Video
- Geometry-aware 4D Video Generation for Robot Manipulation cites Learning Transferable Visual Models From Natural Language Supervision
- Geometry-aware 4D Video Generation for Robot Manipulation cites Denoising Diffusion Probabilistic Models
Cited by top-tier papers22
- Puppeteer: Rig and Animate Your 3D ModelsChaoyue Song, Xiu Li, Fan Yang, Zhongcong Xu et al.NeurIPS 2025 · 48 citations
- Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-DistillationSherwin Bahmani, Tianchang Shen, Jiawei Ren, Jiahui Huang et al.ICLR 2026 · 33 citations
- Geometry-aware 4D Video Generation for Robot ManipulationZeyi Liu, Shuang Li, Eric Cousineau, Siyuan Feng et al.ICLR 2026 · 28 citations
- ShapeGen4D: Towards High Quality 4D Shape Generation from VideosJiraphon Yenphraphai, Ashkan Mirzaei, Jianqi Chen, Jiaxu Zou et al.ICLR 2026 · 21 citations
- V2M4: 4D Mesh Animation Reconstruction from a Single Monocular VideoJianqi Chen, Biao Zhang, Xiangjun Tang, Peter WonkaICCV 2025 · 8 citations
Builds on40
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 6,759 citations
Related papers
- Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion ModelsHuan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler et al.CVPR 2024
- Animate3D: Animating Any 3D Model with Multi-view Video DiffusionYanqin Jiang, Chaohui Yu, Chenjie Cao, Fan Wang et al.NeurIPS 2024 · 65 citations
- Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion ModelsHanwen Liang, Yuyang Yin, Dejia Xu, Hanxue Liang et al.NeurIPS 2024 · 116 citations
- 4Diffusion: Multi-view Video Diffusion Model for 4D GenerationHaiyu Zhang, Xinyuan Chen, Yaohui Wang, Xihui Liu et al.NeurIPS 2024 · 119 citations
- CAD : Photorealistic 3D Generation via Adversarial DistillationZiyu Wan, Despoina Paschalidou, Ian Huang, Hongyu Liu et al.CVPR 2024 · 3 citations
