TokenHand: Discrete Token Representation for Efficient Hand Mesh Reconstruction
Xinguo He, Yixin Shen, Rahul Chaudhari
Abstract
Hand mesh reconstruction has attracted growing attention in recent years. Despite significant progress, existing methods often struggle to balance reconstruction quality and inference efficiency. In this work, we propose TokenHand, a novel framework for single-view 3D hand mesh reconstruction that achieves both high accuracy and real-time inference. Our method represents a 3D hand model using M discrete tokens, each describing a specific sub-structure of the hand. This compositional representation enables efficient modeling with minimal reconstruction error. Furthermore, we reformulate hand mesh reconstruction as a classification problem rather than a regression task. Specifically, a classifier predicts the categories of the M tokens from an input image, and a pre-trained decoder network subsequently reconstructs the 3D hand mesh from the predicted tokens without any post-processing. Extensive experiments demonstrate that TokenHand achieves comparable or superior performance to existing methods across standard benchmarks, while maintaining high efficiency in practical scenarios.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on34
- MLP-Mixer: An all-MLP Architecture for VisionIlya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer et al.NeurIPS 2021 · 3,862 citations
- MetaFormer is Actually What You Need for VisionWeihao Yu, Mi Luo, Pan Zhou, Chenyang Si et al.CVPR 2022 · 1,114 citations
- FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB ImagesChristian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan C. Russell et al.ICCV 2019 · 493 citations
- Mesh GraphormerKevin Lin, Lijuan Wang, Zicheng LiuICCV 2021 · 399 citations
- FastViT: A Fast Hybrid Vision Transformer using Structural ReparameterizationPavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel et al.ICCV 2023 · 341 citations
Related papers
- A Simple Baseline for Efficient Hand Mesh ReconstructionZhishan Zhou, Shihao Zhou, Zhi Lv, Minqiang Zou et al.CVPR 2024 · 18 citations
- End-to-End Hand Mesh Recovery From a Monocular RGB ImageXiong Zhang, Qiang Li, Hong Mo, Wenbo Zhang et al.ICCV 2019 · 248 citations
- MaskHand: Generative Masked Modeling for Robust Hand Mesh Reconstruction in the WildMuhammad Usama Saleem, Ekkasit Pinyoanuntapong, Mayur Jagdishbhai Patel, Hongfei Xue et al.ICCV 2025 · 2 citations
- TORE: Token Reduction for Efficient Human Mesh Recovery with TransformerZhiyang Dou, Qingxuan Wu, Cheng Lin, Zeyu Cao et al.ICCV 2023 · 56 citations
- H2ONet: Hand-Occlusion-and-Orientation-Aware Network for Real-Time 3D Hand Mesh ReconstructionHao Xu, Tianyu Wang, Xiao Tang, Chi-Wing FuCVPR 2023
