ASTrA: Adversarial Self-supervised Training with Adaptive-Attacks
Prakash Chandra Chhipa, Gautam Vashishtha, Settur Jithamanyu, Rajkumar Saini, Mubarak Shah, Marcus Liwicki
Abstract
Existing self-supervised adversarial training (self-AT) methods rely on handcrafted adversarial attack strategies for PGD attacks, which fail to adapt to the evolving learning dynamics of the model and do not account for instancespecific characteristics of images. This results in sub-optimal adversarial robustness and limits the alignment between clean and adversarial data distributions.
To address this, we propose ASTrA (Adversarial Self-supervised Training with Adaptive-Attacks), a novel framework introducing a learnable, self-supervised attack strategy network that autonomously discovers optimal attack parameters through exploration-exploitation in a single training episode. ASTrA leverages a reward mechanism based on contrastive loss, optimized with REINFORCE, enabling adaptive attack strategies without labeled data or additional hyperparameters. We further introduce a mixed contrastive objective to align the distribution of clean and adversarial examples in representation space. ASTrA achieves state-of-the-art results on CIFAR10, CIFAR100, and STL10 while integrating seamlessly as a plug-and-play module for other self-AT methods. AS-TrA shows scalability to larger datasets, demonstrates strong semi-supervised performance, and is resilient to robust overfitting, backed by explainability analysis on optimal attack strategies. Project page for source code and other details at https://prakashchhipa.github.io/projects/ASTrA.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution cites A Simple Framework for Contrastive Learning of Visual Representations
- Whitening for Self-Supervised Representation Learning cites A Simple Framework for Contrastive Learning of Visual Representations
Builds on16
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- On Adaptive Attacks to Adversarial Example DefensesFlorian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander MadryNeurIPS 2020 · 1,026 citations
- Fine-Tuning can Distort Pretrained Features and Underperform Out-of-DistributionAnanya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma et al.ICLR 2022 · 911 citations
- Improving Adversarial Robustness Requires Revisiting Misclassified ExamplesYisen Wang, Difan Zou, Jinfeng Yi, James Bailey et al.ICLR 2020 · 829 citations
- Whitening for Self-Supervised Representation LearningAleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, Nicu SebeICML 2021 · 378 citations
Related papers
- LAS-AT: Adversarial Training with Learnable Attack StrategyXiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma et al.CVPR 2022 · 140 citations
- Rethinking the Effect of Data Augmentation in Adversarial Contrastive LearningRundong Luo, Yifei Wang, Yisen WangICLR 2023 · 2 citations
- 3SAT: A Simple Self-Supervised Adversarial Training FrameworkJiang Fang, Haonan He, Jiyan Sun, Jiadong Fu et al.AAAI 2025 · 3 citations
- Adversarial Self-Supervised Contrastive LearningMinseon Kim, Jihoon Tack, Sung Ju HwangNeurIPS 2020 · 294 citations
- Robust Pre-Training by Adversarial Contrastive LearningZiyu Jiang, Tianlong Chen, Ting Chen, Zhangyang WangNeurIPS 2020 · 284 citations
