Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Figure 1 : We propose an approach for end-to-end neural network training with mesh supervision that is obtained through an automated data collection method. We process a large collection of YouTube videos and analyze them with 2D hand keypoint detector followed by parametric model fitting (right side). The fitting results are used as a supervisory signal ('mesh loss') for a feed-forward network with a mesh convolutional decoder tasked with recovering a 3D hand mesh at its output (left side).