VDFE: Difference-Aware 3D Scene Editing with Non-Intrusive Video Diffusion Priors for Multi-View Consistency and Efficiency
Chao Zhang, Fang Liu, Shuo Li, Yang Liu, Jiahao Wang, Xinyan Huang, Lingling Li, Puhua Chen, Xu Liu, Wenping Ma, Siqi Yu
Abstract
Text-driven 3D editing, enabled by advancements in 3D reconstruction techniques such as NeRF and 3D Gaussian Splatting, aims to provide intuitive scene customization. However, existing methods frequently exhibit limitations in controllability and consistency. To address these shortcomings, we propose VDFE, a difference-aware 3D scene editing method based on non-intrusive utilization of pre-trained video diffusion priors, which integrates Optimal Control Guided Flow Editing (FlowOCE), Decoupled Flow Difference (DFD), and Difference-Aware Gaussians Editing (DAGE). Specifically, FlowOCE treats the editing process as an optimal control problem, optimizing a noise-free editing trajectory to minimize unintended modifications in non-target region; DFD precisely locates editing region by analyzing flow differences, which supplies priors for the subsequent optimization process; and DAGE leverages precise localization to selectively update 3D Gaussians for efficient and precise refinement. Extensive experiments demonstrate that our method significantly outperforms existing methods in both qualitative and quantitative evaluations, achieving state-of-the-art (SOTA) performance.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on38
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 11,724 citations
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
Related papers
- Edit3D: Elevating 3D Scene Editing with Attention-Driven Multi-Turn InteractivityPeng Zhou, Dunbo Cai, Yujian Du, Runqing Zhang et al.ACM MM 2024 · 3 citations
- GaussianEditor: Editing 3D Gaussians Delicately with Text InstructionsJunjie Wang, Jiemin Fang, Xiaopeng Zhang, Lingxi Xie et al.CVPR 2024 · 65 citations
- Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion TransformerDong In Lee, Hyungjun Doh, Seunggeun Chi, Runlin Duan et al.CVPR 2026 · 3 citations
- D2Gaussian: Dynamic Control with Discretized 3D View Modeling for Text-Driven 3D Gaussian Splatting EditingYefei Sheng, Jie Wang, Ming Tao, Bing-Kun BaoACM MM 2025 · 1 citation
- 3D Gaussian Editing with A Single ImageGuan Luo, Tian-Xing Xu, Ying-Tian Liu, Xiaoxiong Fan et al.ACM MM 2024 · 7 citations
