Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videos
Ziqian Bai, Feitong Tan, Zeng Huang, Kripasindhu Sarkar, Danhang Tang, Di Qiu, Abhimitra Meka, Ruofei Du, Mingsong Dou, Sergio Orts-Escolano, Rohit Pandey, Ping Tan
Abstract
Figure 1 . Our technique builds a 3D avatar representation of a person using just a single short monocular RGB video (e.g., 1-2 minutes). We leverage a 3DMM to track the user's expressions. By anchoring a neural radiance field to the 3DMM geometry, we generate a volumetric photorealistic 3D avatar that can be rendered with user-defined expression and viewpoint. Note that our method works well on challenging materials, e.g., hair and dramatic expressions. Please see our webpage augmentedperception.github.io/monoavatar for more results.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 095b7dad-02cf-4418-8026-5ec500dbf6a0Cited by top-tier papers27
- Generalizable and Animatable Gaussian Head AvatarXuangeng Chu, Tatsuya HaradaNeurIPS 2024 · 115 citations
- SplattingAvatar: Realistic Real-Time Human Avatars With Mesh-Embedded Gaussian SplattingZhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang et al.CVPR 2024 · 92 citations
- MonoGaussianAvatar: Monocular Gaussian Point-based Head AvatarYufan Chen, Lizhen Wang, Qijing Li, Hongjiang Xiao et al.SIGGRAPH 2024 · 85 citations
- GPAvatar: Generalizable and Precise Head Avatar from Image(s)Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang et al.ICLR 2024 · 63 citations
- VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head ReenactmentPhong Tran, Egor Zakharov, Long-Nhat Ho, Anh Tuan Tran et al.CVPR 2024 · 15 citations
Builds on19
- FaceForensics++: Learning to Detect Manipulated Facial ImagesAndreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess et al.ICCV 2019 · 2,966 citations
- Neural Sparse Voxel FieldsLingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua et al.NeurIPS 2020 · 1,535 citations
- Nerfies: Deformable Neural Radiance FieldsKeunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz et al.ICCV 2021 · 1,442 citations
- PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human DigitizationShunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima et al.ICCV 2019 · 1,411 citations
- Few-Shot Adversarial Learning of Realistic Neural Talking Head ModelsEgor Zakharov, Aliaksandra Shysheya, Egor Burkov, Victor S. LempitskyICCV 2019 · 687 citations
Related papers
- Instant Volumetric Head AvatarsWojciech Zielonka, Timo Bolkart, Justus ThiesCVPR 2023
- I M Avatar: Implicit Morphable Head Avatars from VideosYufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen et al.CVPR 2022 · 169 citations
- FlashAvatar: High-Fidelity Head Avatar with Efficient Gaussian EmbeddingJun Xiang, Xuan Gao, Yudong Guo, Juyong ZhangCVPR 2024 · 51 citations
- AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural VoxelsYuelang Xu, Lizhen Wang, Xiaochen Zhao, Hongwen Zhang et al.SIGGRAPH 2023 · 69 citations
- InstantAvatar: Learning Avatars from Monocular Video in 60 SecondsTianjian Jiang, Xu Chen, Jie Song, Otmar HilligesCVPR 2023
