NeuFace: Realistic 3D Neural Face Rendering from Multi-View Images

Realistic face rendering from multi-view images is beneficial to various computer vision and graphics applications. Due to complex spatially-varying reflectance properties and geometry characteristics of faces, however, it remains challenging to recover 3D facial representations both faithfully and efficiently in the current studies. This paper presents a novel 3D face rendering model, namely NeuFace, to learn accurate and physically-meaningful underlying 3D representations by neural rendering techniques. It naturally incorporates the neural BRDFs into physically based rendering, capturing sophisticated facial geometry and appearance clues in a collaborative manner. Specifically, we introduce an approximated BRDF integration and a simple yet new low-rank prior, which effectively lower the ambiguities and boost the performance of the facial BRDFs. Extensive experiments are performed to demonstrate the superiority of NeuFace in human face rendering, along with a decent generalization ability to common objects. Code is released at NeuFace.