High-Res Facial Appearance Capture from Polarized Smartphone Images

Figure 1 . Our method obtains high-resolution skin textures from two RGB input sequences captured with polarization foils attached to a smartphone. The core idea is to separate the skin's diffuse and specular response by capturing one cross-polarized and one parallelpolarized sequence. We recover an accurate geometry with multi-view stereo, fit a parametric head model, and employ a differentiable rendering strategy to recover 4K diffuse albedo, specular gain and normal maps. These can be used with off-the-shelf rendering software, such as Blender, to produce photo-realistic images from novel views, under novel illumination and with subsurface scattering (SSS).