Adaptive Wasserstein Hourglass for Weakly Supervised RGB 3D Hand Pose Estimation

The deficiency of labeled training data is one of the bottlenecks in 3D hand pose estimation from monocular RGB images. Synthetic datasets have a large number of images with precise annotations, but their obvious difference with real-world datasets limits the generalization ability. Few efforts have been made to bridge the gap between the two domains in terms of their large differences. In this paper, we propose a domain adaptation method called Adaptive Wasserstein Hourglass for weakly-supervised 3D hand pose estimation to close the large gap between synthetic and real-world datasets flexibly. Adaptive Wasserstein Hourglass utilizes a feature similarity metric to identify the differences and explore the common features (e.g., hand structure) of the two datasets. Common features are drawn close adaptively during the training, whereas domain-specific features retain the differences. Learning common features helps the network in focusing on pose-related information, whereas maintaining domain-specific features reduces the optimization difficulty when closing the big gap between two domains. Extensive evaluations on two benchmark datasets demonstrate that our method succeeds in distinguishing different features and achieves optimal results when compared with state-of-the-art 3D pose estimation approaches and domain adaptation methods.