Harnessing the Power of SVD: An SVA Module for Enhanced Signal Classification
Lei Zhai, Shuyuan Yang, Yitong Li, Zhixi Feng, Zhihao Chang, Quanwei Gao
Abstract
Deep learning methods have achieved outstanding performance in various signal tasks. However, due to degraded signals in real electromagnetic environment, it is crucial to seek methods that can improve the representation of signal features. In this paper, a Singular Value decomposition-based Attention, SVA is proposed to explore structure of signal data for adaptively enhancing intrinsic feature. Using a deep neural network as a base model, SVA performs feature semantic subspace learning through a decomposition layer and combines it with an attention layer to achieve adaptive enhancement of signal features. Moreover, we consider the gradient explosion problem brought by SVA and optimize SVA to improve the stability of training. Extensive experimental results demon-strate that applying SVA to a generalized classification model can significantly improve its ability in representations, making its recognition performance competitive with, or even better than, the state-of-the-art task-specific models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows cites Exploring Self-Attention for Image Recognition
- Vision Transformer with Deformable Attention cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks cites ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
Builds on5
- Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
- SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural NetworksLingxiao Yang, Ru-Yuan Zhang, Lida Li, Xiaohua XieICML 2021 · 1,593 citations
- Vision Transformer with Deformable AttentionZhuofan Xia, Xuran Pan, Shiji Song, Li Erran Li et al.CVPR 2022 · 835 citations
- Exploring Self-Attention for Image RecognitionHengshuang Zhao, Jiaya Jia, Vladlen KoltunCVPR 2020
- ECA-Net: Efficient Channel Attention for Deep Convolutional Neural NetworksQilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li et al.CVPR 2020
Related papers
- Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in sEMG AnalysisWeiyu Guo, Ziyue Qiao, Ying Sun, Yijie Xu et al.ICML 2025
- Graph Convolutions Enrich the Self-Attention in Transformers!Jeongwhan Choi, Hyowon Wi, Jayoung Kim, Yehjin Shin et al.NeurIPS 2024 · 24 citations
- Generalized and Discriminative Few-Shot Object Detection via SVD-Dictionary EnhancementAming Wu, Suqi Zhao, Cheng Deng, Wei LiuNeurIPS 2021 · 52 citations
- What Do Deep Saliency Models Learn about Visual Attention?Shi Chen, Ming Jiang, Qi ZhaoNeurIPS 2023 · 12 citations
- TapNet: Multivariate Time Series Classification with Attentional Prototypical NetworkXuchao Zhang, Yifeng Gao, Jessica Lin, Chang-Tien LuAAAI 2020 · 363 citations
