Lune

AAAI2024Top-tier venue

Harnessing the Power of SVD: An SVA Module for Enhanced Signal Classification

Lei Zhai, Shuyuan Yang, Yitong Li, Zhixi Feng, Zhihao Chang, Quanwei Gao

Abstract

Deep learning methods have achieved outstanding performance in various signal tasks. However, due to degraded signals in real electromagnetic environment, it is crucial to seek methods that can improve the representation of signal features. In this paper, a Singular Value decomposition-based Attention, SVA is proposed to explore structure of signal data for adaptively enhancing intrinsic feature. Using a deep neural network as a base model, SVA performs feature semantic subspace learning through a decomposition layer and combines it with an attention layer to achieve adaptive enhancement of signal features. Moreover, we consider the gradient explosion problem brought by SVA and optimize SVA to improve the stability of training. Extensive experimental results demon-strate that applying SVA to a generalized classification model can significantly improve its ability in representations, making its recognition performance competitive with, or even better than, the state-of-the-art task-specific models.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Swin Transformer: Hierarchical Vision Transformer using Shifted Windows cites Exploring Self-Attention for Image Recognition
  • Vision Transformer with Deformable Attention cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  • SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks cites ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
  2. SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural NetworksLingxiao Yang, Ru-Yuan Zhang, Lida Li, Xiaohua XieICML 2021 · 1,593 citations
  3. Vision Transformer with Deformable AttentionZhuofan Xia, Xuran Pan, Shiji Song, Li Erran Li et al.CVPR 2022 · 835 citations
  4. Exploring Self-Attention for Image RecognitionHengshuang Zhao, Jiaya Jia, Vladlen KoltunCVPR 2020
  5. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural NetworksQilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li et al.CVPR 2020

Related papers

  1. Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in sEMG AnalysisWeiyu Guo, Ziyue Qiao, Ying Sun, Yijie Xu et al.ICML 2025
  2. Graph Convolutions Enrich the Self-Attention in Transformers!Jeongwhan Choi, Hyowon Wi, Jayoung Kim, Yehjin Shin et al.NeurIPS 2024 · 24 citations
  3. Generalized and Discriminative Few-Shot Object Detection via SVD-Dictionary EnhancementAming Wu, Suqi Zhao, Cheng Deng, Wei LiuNeurIPS 2021 · 52 citations
  4. What Do Deep Saliency Models Learn about Visual Attention?Shi Chen, Ming Jiang, Qi ZhaoNeurIPS 2023 · 12 citations
  5. TapNet: Multivariate Time Series Classification with Attentional Prototypical NetworkXuchao Zhang, Yifeng Gao, Jessica Lin, Chang-Tien LuAAAI 2020 · 363 citations
Dusk over the sea between two cliffs drawn in fine vertical lines