Lune

AAAI2022Top-tier venue

Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis

Taha Belkhouja, Yan Yan, Janardhan Rao Doppa

Abstract

Despite the success of deep neural networks (DNNs) for real-world applications over time-series data such as mobile health, little is known about how to train robust DNNs for time-series domain due to its unique characteristics compared to images and text data. In this paper, we fill this gap by proposing a novel algorithmic framework referred as RObust Training for Time-Series (RO-TS) to create robust deep models for time-series classification tasks. Specifically, we formulate a min-max optimization problem over the model parameters by explicitly reasoning about the robustness criteria in terms of additive perturbations to timeseries inputs measured by the global alignment kernel (GAK) based distance. We also show the generality and advantages of our formulation using the summation structure over time-series alignments by relating both GAK and dynamic time warping (DTW). This problem is an instance of a family of compositional min-max optimization problems, which are challenging and open with unclear theoretical guarantee. We propose a principled stochastic compositional alternating gradient descent ascent (SCAGDA) algorithm for this family of optimization problems. Unlike traditional methods for time-series that require approximate computation of distance measures, SCAGDA approximates the GAK based distance on-the-fly using a moving average approach. We theoretically analyze the convergence rate of SCAGDA and provide strong theoretical support for the estimation of GAK based distance. Our experiments on realworld benchmarks demonstrate that RO-TS creates more robust deep models when compared to adversarial training using prior methods that rely on data augmentation or new definitions of loss functions. We also demonstrate the importance of GAK for time-series data over the Euclidean distance. The source code of RO-TS algorithms is available at https://github.com/tahabelkhouja/Robust-Trainingfor-Time-Series

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time SeriesQianwen Meng, Hangwei Qian, Yong Liu, Lizhen Cui et al.AAAI 2023 · 54 citations

Builds on3

  1. Towards Evaluating the Robustness of Neural NetworksNicholas Carlini, David A. WagnerS&P 2017 · 9,786 citations
  2. Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax ProblemsJunchi Yang, Negar Kiyavash, Niao HeNeurIPS 2020 · 136 citations
  3. Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max OptimizationYan Yan, Yi Xu, Qihang Lin, Wei Liu et al.NeurIPS 2020 · 70 citations

Related papers

  1. Amata: An Annealing Mechanism for Adversarial Training AccelerationNanyang Ye, Qianxiao Li, Xiao-Yun Zhou, Zhanxing ZhuAAAI 2021 · 18 citations
  2. Compositional Training for End-to-End Deep AUC MaximizationZhuoning Yuan, Zhishuai Guo, Nitesh V. Chawla, Tianbao YangICLR 2022 · 34 citations
  3. Improving Adversarial Robustness by Putting More Regularizations on Less Robust SamplesDongyoon Yang, Insung Kong, Yongdai KimICML 2023 · 15 citations
  4. Joint-Label Learning by Dual Augmentation for Time Series ClassificationQianli Ma, Zhenjing Zheng, Jiawei Zheng, Sen Li et al.AAAI 2021 · 17 citations
  5. Functional Virtual Adversarial Training for Semi-Supervised Time Series ClassificationQingyi Pan, Yicheng LiNeurIPS 2025
Dusk over the sea between two cliffs drawn in fine vertical lines