Adversarial Robust ViT-Based Automatic Modulation Recognition in Practical Deep Learning-Based Wireless Systems

Advanced wireless communication systems adopt deep learning (DL) approaches to achieve automatic modulation recognition (AMR) for spectrum monitoring and management, especially in the spectrum bands supporting diverse co-existing wireless protocols. In practical wireless environments, wireless signals can easily get compromised by malicious noise, intentional interference, and adversarial attacks, reducing the effectiveness of AMR. By exploiting DL model vulnerabilities, an undetectable perturbation added to the wireless signal can cause misclassification, resutling in serious consequences including decoding errors, throughput degradation, and communication disruption. Facing the limitations of existing works on defending against wireless adversarial attacks, this work innovates the Transformer model to design an adversarial robust AMR driven by exploring temporal correlation in time-sequence wireless signals. Instead of directly applying the Vision Transformer (ViT), we first innovate a feature extraction module specifically for radio frequency (RF) signals from both the time and frequency domains, together with an adaptive positional embedding to the Transformer encoder for enhancing AMR accuracy. To mitigate the noise effect in practical wireless communication, we then propose a noise-adaptive adversarial training scheme on the developed Transformer-based model using adversarial examples crafted by white-box attackers. To show the scheme's efficiency, effectiveness, and robustness, our proposed design has been thoroughly evaluated via a self-collected real-world dataset consisting of over 30 million wireless signal data samples with 21 modulation schemes in both indoor and outdoor scenarios. Our results reach a maximum accuracy of 94.17% in AMR classification and 71.2 % under adversarial attacks. Besides, for the first time, we demonstrate the robustness of our design under a real wireless adversarial attack in real-time. Datasets and code available in https://github.com/coulsonlee/Robust-ViT-for-AMR-SP2025.