Learning Complete 3D Morphable Face Models From Images and Videos
Mallikarjun B. R., Ayush Tewari, Hans-Peter Seidel, Mohamed A. Elgharib, Christian Theobalt
Abstract
Figure 1. We present a method for learning complete 3D morphable models of faces from videos and images. We show visualizations of the learned models on the right. Faces in each direction of indicated arrows is obtained by linearly scaling individual component of respective models. Identity geometry captures variations in the face shape (second column), lips (top left to bottom right) and jaw (top right to bottom left), while expressions capture variations due to mouth opening (second row), smile (second column) and eye movement (top right to bottom left). Albedo/Reflectance spans a variety of skin color (second column), eye color (top right to bottom left) and gender specific features such as facial hair and make-up (second row).
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers23
- ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural RepresentationsMingwu Zheng, Hongyu Yang, Di Huang, Liming ChenCVPR 2022 · 60 citations
- 3D Facial Expressions through Analysis-by-Neural-SynthesisGeorge Retsinas, Panagiotis Paraskevas Filntisis, Radek Danecek, Victoria Fernández Abrevaya et al.CVPR 2024 · 26 citations
- 3D Face Reconstruction with the Geometric Guidance of Facial Part SegmentationZidu Wang, Xiangyu Zhu, Tianshuo Zhang, Baiqin Wang et al.CVPR 2024 · 23 citations
- Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices?Cho-Ying Wu, Chin-Cheng Hsu, Ulrich NeumannCVPR 2022 · 16 citations
- Single-Shot Implicit Morphable Faces with Consistent Texture ParameterizationConnor Z. Lin, Koki Nagano, Jan Kautz, Eric R. Chan et al.SIGGRAPH 2023 · 14 citations
Builds on3
- Learning Formation of Physically-Based Face AttributesRuilong Li, Karl Bladin, Yajie Zhao, Chinmay Chinara et al.CVPR 2020
- Uncertainty-Aware Mesh Decoder for High Fidelity 3D Face ReconstructionGun-Hee Lee, Seong-Whan LeeCVPR 2020
- Towards High-Fidelity 3D Face Reconstruction From In-the-Wild Images Using Graph Convolutional NetworksJiangke Lin, Yi Yuan, Tianjia Shao, Kun ZhouCVPR 2020
Related papers
- i3DMM: Deep Implicit 3D Morphable Model of Human HeadsTarun Yenamandra, Ayush Tewari, Florian Bernard, Hans-Peter Seidel et al.CVPR 2021
- I M Avatar: Implicit Morphable Head Avatars from VideosYufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen et al.CVPR 2022 · 169 citations
- Learning a 3D Morphable Face Reflectance Model from Low-Cost DataYuxuan Han, Zhibo Wang, Feng XuCVPR 2023
- RigNeRF: Fully Controllable Neural 3D PortraitsShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli, Eli Shechtman et al.CVPR 2022 · 117 citations
- Implicit Neural Head Synthesis via Controllable Local Deformation FieldsChuhan Chen, Matthew O'Toole, Gaurav Bharaj, Pablo GarridoCVPR 2023
