PRISM: Pre-training RF Signals in Sparsity-aware Masked Autoencoders

This paper introduces a novel paradigm for learning-based RF sensing, termed Pre-training RF signals In Sparsity-aware Masked autoencoders (PRISM), which shifts the RF sensing paradigm from supervised training on limited annotated datasets to unsupervised pre-training on large-scale unannotated datasets, followed by fine-tuning with a small annotated dataset. PRISM leverages a carefully designed sparsity-aware masking strategy to predict missing contents by masking a portion of RF signals, resulting in an efficient pre-training framework that significantly reduces computation and memory resources. This addresses the major challenges posed by large-scale and high-dimensional RF datasets, where memory consumption and computation speed are critical factors. We demonstrate PRISM’s excellent generalization performance across diverse RF sensing tasks by evaluating it on three typical scenarios: human silhouette segmentation, 3D pose estimation, and gesture recognition, involving two general RF devices, radar and WiFi. The experimental results provide strong evidence for the effectiveness of PRISM as a robust learning-based solution for large-scale RF sensing applications.