Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild
Dominik Kulon, Riza Alp Güler, Iasonas Kokkinos, Michael M. Bronstein, Stefanos Zafeiriou
Abstract
Figure 1 : We propose an approach for end-to-end neural network training with mesh supervision that is obtained through an automated data collection method. We process a large collection of YouTube videos and analyze them with 2D hand keypoint detector followed by parametric model fitting (right side). The fitting results are used as a supervisory signal ('mesh loss') for a feed-forward network with a mesh convolutional decoder tasked with recovering a 3D hand mesh at its output (left side).
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 2fa1f6d7-8f6a-4028-b5cb-f8bede82608eCited by top-tier papers73
- FastViT: A Fast Hybrid Vision Transformer using Structural ReparameterizationPavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel et al.ICCV 2023 · 341 citations
- Reconstructing Hand-Object Interactions in the WildZhe Cao, Ilija Radosavovic, Angjoo Kanazawa, Jitendra MalikICCV 2021 · 184 citations
- CPF: Learning a Contact Potential Field to Model the Hand-Object InteractionLixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu et al.ICCV 2021 · 170 citations
- Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose EstimationShreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, Vincent LepetitCVPR 2022 · 155 citations
- HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation NetworkJoonKyu Park, Yeonguk Oh, Gyeongsik Moon, Hongsuk Choi et al.CVPR 2022 · 116 citations
Builds on6
- Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the LoopNikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas DaniilidisICCV 2019 · 1,139 citations
- Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional NetworksYujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai et al.ICCV 2019 · 504 citations
- FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB ImagesChristian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan C. Russell et al.ICCV 2019 · 493 citations
- End-to-End Hand Mesh Recovery From a Monocular RGB ImageXiong Zhang, Qiang Li, Hong Mo, Wenbo Zhang et al.ICCV 2019 · 248 citations
- Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and GenerationGiorgos Bouritsas, Sergiy Bokhnyak, Stylianos Ploumpis, Stefanos Zafeiriou et al.ICCV 2019 · 187 citations
Related papers
- Model-Based 3D Hand Reconstruction via Self-Supervised LearningYujin Chen, Zhigang Tu, Di Kang, Linchao Bao et al.CVPR 2021
- Overcoming the TradeOff between Accuracy and Plausibility in 3D Hand Shape ReconstructionZiwei Yu, Chen Li, Linlin Yang, Xiaoxu Zheng et al.CVPR 2023
- Learning Monocular 3D Reconstruction of Articulated Categories From MotionFilippos Kokkinos, Iasonas KokkinosCVPR 2021
- HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation From a Single Depth MapJameel Malik, Ibrahim Abdelaziz, Ahmed Elhayek, Soshi Shimada et al.CVPR 2020
- Self-Supervised Human Depth Estimation From Monocular VideosFeitong Tan, Hao Zhu, Zhaopeng Cui, Siyu Zhu et al.CVPR 2020
