OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation
Yuchen Lin, Chenguo Lin, Jianjin Xu, Yadong Mu
Abstract
Recently, significant advancements have been made in the reconstruction and generation of 3D assets, including static cases and those with physical interactions. To recover the physical properties of 3D assets, existing methods typically assume that all materials belong to a specific predefined category (e.g., elasticity). However, such assumptions ignore the complex composition of multiple heterogeneous objects in real scenarios and tend to render less physically plausible animation given a wider range of objects. We propose OMNIPHYSGS for synthesizing a physics-based 3D dynamic scene composed of more general objects. A key design of OMNIPHYSGS is treating each 3D asset as a collection of constitutive 3D Gaussians. For each Gaussian, its physical material is represented by an ensemble of 12 physical domain-expert sub-models (rubber, metal, honey, water, etc.), which greatly enhances the flexibility of the proposed model. In the implementation, we define a scene by user-specified prompts and supervise the estimation of material weighting factors via a pretrained video diffusion model. Comprehensive experiments demonstrate that OMNIPHYSGS achieves more general and realistic physical dynamics across a broader spectrum of materials, including elastic, viscoelastic, plastic, and fluid substances, as well as interactions between different materials. Our method surpasses existing methods by approximately 3% to 16% in metrics of visual quality and text alignment.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
25 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation cites Learning Transferable Visual Models From Natural Language Supervision
- PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesis cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesis cites Learning Transferable Visual Models From Natural Language Supervision
- PhysInOne: Visual Physics Learning and Reasoning in One Suite cites Video Diffusion Models
- PhysInOne: Visual Physics Learning and Reasoning in One Suite cites VoMP: Predicting Volumetric Mechanical Property Fields
- VoMP: Predicting Volumetric Mechanical Property Fields cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- VoMP: Predicting Volumetric Mechanical Property Fields cites Learning Transferable Visual Models From Natural Language Supervision
- MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- Learning 3D-Gaussian Simulators from RGB Videos cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation cites ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers25
- MoVieS: Motion-Aware 4D Dynamic View Synthesis in One SecondChenguo Lin, Yuchen Lin, Panwang Pan, Yifan Yu et al.CVPR 2026 · 38 citations
- VoMP: Predicting Volumetric Mechanical Property FieldsRishit Dagli, Donglai Xiang, Vismay Modi, Charles Loop et al.ICLR 2026 · 13 citations
- PhysGM: Large Physical Gaussian Model for Feed-Forward 4D SynthesisChunji Lv, Zequn Chen, Donglin Di, Weinan Zhang et al.CVPR 2026 · 9 citations
- PhysInOne: Visual Physics Learning and Reasoning in One SuiteSiyuan Zhou, Hejun Wang, Hu Cheng, Jinxi Li et al.CVPR 2026 · 9 citations
- Learning 3D-Gaussian Simulators from RGB VideosMikel Zhobro, Andreas René Geist, Georg MartiusICML 2026 · 8 citations
Builds on26
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- Video Diffusion ModelsJonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan et al.NeurIPS 2022 · 2,948 citations
- ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score DistillationZhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao et al.NeurIPS 2023 · 1,498 citations
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content CreationJiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu et al.ICLR 2024 · 955 citations
Related papers
- Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene SimulationZhuoman Liu, Weicai Ye, Yan Luximon, Pengfei Wan et al.CVPR 2025
- M-PhyGs: Multi-Material Object Dynamics from VideoNorika Wada, Kohei Yamashita, Ryo Kawahara, Ko NishinoCVPR Findings 2026
- PhysGaia: A Physics-aware Benchmark with Multi-Body Interactions for Dynamic Novel View SynthesisMijeong Kim, Gunhee Kim, Jungyoon Choi, Wonjae Roh et al.CVPR 2026 · 2 citations
- Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and RenderingYutao Feng, Xiang Feng, Yintong Shang, Ying Jiang et al.CVPR 2025
- PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property EstimationSamarth Chopra, Jing Liang, Gershom Seneviratne, Dinesh ManochaCVPR 2026 · 6 citations
