D-IF: Uncertainty-aware Human Digitization via Implicit Distribution Field
Xueting Yang, Yihao Luo, Yuliang Xiu, Wei Wang, Hao Xu, Zhaoxin Fan
Abstract
Realistic virtual humans play a crucial role in numerous industries, such as metaverse, intelligent healthcare, and self-driving simulation. But creating them on a large scale with high levels of realism remains a challenge. The utilization of deep implicit function sparks a new era of image-based 3D clothed human reconstruction, enabling pixel-aligned shape recovery with fine details. Subsequently, the vast majority of works locate the surface by regressing the deterministic implicit value for each point. However, should all points be treated equally regardless of their proximity to the surface? In this paper, we propose replacing the implicit value with an adaptive uncertainty distribution, to differentiate between points based on their distance to the surface. This simple "value ⇒ distribution" transition yields significant improvements on nearly all the baselines. Furthermore, qualitative results demonstrate that the models trained using our uncertainty distribution loss, can capture more intricate wrinkles, and realistic limbs. Code and models are available for research purposes at github.com/psyai-net/D-IF release.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers19
- HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure PriorsPanwang Pan, Zhuo Su, Chenguo Lin, Zhen Fan et al.NeurIPS 2024 · 76 citations
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion ModelsYuxuan Xue, Xianghui Xie, Riccardo Marin, Gerard Pons-MollNeurIPS 2024 · 49 citations
- SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human ReconstructionZechuan Zhang, Zongxin Yang, Yi YangCVPR 2024 · 44 citations
- HumanRef: Single Image to 3D Human Generation via Reference-Guided DiffusionJingbo Zhang, Xiaoyu Li, Qi Zhang, Yanpei Cao et al.CVPR 2024 · 15 citations
- HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric ModelsYifan Yang, Dong Liu, Shuhai Zhang, Zeshuai Deng et al.CVPR 2024 · 11 citations
Builds on21
- PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human DigitizationShunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima et al.ICCV 2019 · 1,411 citations
- Multi-Garment Net: Learning to Dress 3D People From ImagesBharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, Gerard Pons-MollICCV 2019 · 447 citations
- HuMoR: 3D Human Motion Model for Robust Pose EstimationDavis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang et al.ICCV 2021 · 398 citations
- Tex2Shape: Detailed Full Human Body Geometry From a Single ImageThiemo Alldieck, Gerard Pons-Moll, Christian Theobalt, Marcus A. MagnorICCV 2019 · 343 citations
- Dense Depth Priors for Neural Radiance Fields from Sparse Input ViewsBarbara Roessle, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan et al.CVPR 2022 · 319 citations
Related papers
- Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human ReconstructionTong He, John P. Collomosse, Hailin Jin, Stefano SoattoNeurIPS 2020 · 203 citations
- ECON: Explicit Clothed humans Optimized via Normal integrationYuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas et al.CVPR 2023
- DIFu: Depth-Guided Implicit Function for Clothed Human ReconstructionDae-Young Song, HeeKyung Lee, Jeongil Seo, Donghyeon ChoCVPR 2023
- DLCA-Recon: Dynamic Loose Clothing Avatar Reconstruction from Monocular VideosChunjie Luo, Fei Luo, Yusen Wang, Enxu Zhao et al.AAAI 2024 · 5 citations
- ARCH: Animatable Reconstruction of Clothed HumansZeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li et al.CVPR 2020
