Wi-Hand: 3D Hand Mesh Construction Using WiFi

This paper presents Wi-Hand, a 3D hand construction system using WiFi. Our system leverages WiFi devices commonly available in smart home environments and is capable of operating under non-line-of-sight (NLoS) conditions, offering greater flexibility and ease of deployment compared to traditional computer vision and wearable-based approaches. In specific, our system leverages the spatial information of the reflected WiFi signals to mitigate the dynamic interference in indoor environments and extract the signal reflections of the target subject. We further utilize Doppler velocity information to distinguish WiFi signals reflected from hand motions from those of the human body, as they exhibit different movement speed characteristics. Moreover, our system leverages two-dimensional angle of arrival (2D AoA) information to represent the shape and deformations of the target hand for 3D hand mesh construction. In addition, we leverage the spatial-temporal coherence of the hand to handle the missing parts of the finger due to their weak and specular reflections. Finally, a deep learning model is utilized to digitize 2D AoA frames of the target hand into the 3D hand mesh. We conduct extensive experiments involving multiple participants across diverse environments. The results demonstrate that our system accurately reconstructs 3D hand meshes for free-form hand activities, achieving an average vertex error of 1.05cm. Additionally, we evaluate the performance of our system under various conditions, including different user positions, dynamic interference, hand shape diversity, randomly ordered gestures, and temporally discontinuous hand activities. These evaluations collectively demonstrate that Wi-Hand can effectively and robustly leverage WiFi signals to construct accurate 3D hand meshes.