Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis
Taha Belkhouja, Yan Yan, Janardhan Rao Doppa
Abstract
Despite the success of deep neural networks (DNNs) for real-world applications over time-series data such as mobile health, little is known about how to train robust DNNs for time-series domain due to its unique characteristics compared to images and text data. In this paper, we fill this gap by proposing a novel algorithmic framework referred as RObust Training for Time-Series (RO-TS) to create robust deep models for time-series classification tasks. Specifically, we formulate a min-max optimization problem over the model parameters by explicitly reasoning about the robustness criteria in terms of additive perturbations to timeseries inputs measured by the global alignment kernel (GAK) based distance. We also show the generality and advantages of our formulation using the summation structure over time-series alignments by relating both GAK and dynamic time warping (DTW). This problem is an instance of a family of compositional min-max optimization problems, which are challenging and open with unclear theoretical guarantee. We propose a principled stochastic compositional alternating gradient descent ascent (SCAGDA) algorithm for this family of optimization problems. Unlike traditional methods for time-series that require approximate computation of distance measures, SCAGDA approximates the GAK based distance on-the-fly using a moving average approach. We theoretically analyze the convergence rate of SCAGDA and provide strong theoretical support for the estimation of GAK based distance. Our experiments on realworld benchmarks demonstrate that RO-TS creates more robust deep models when compared to adversarial training using prior methods that rely on data augmentation or new definitions of loss functions. We also demonstrate the importance of GAK for time-series data over the Euclidean distance. The source code of RO-TS algorithms is available at https://github.com/tahabelkhouja/Robust-Trainingfor-Time-Series
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on3
- Towards Evaluating the Robustness of Neural NetworksNicholas Carlini, David A. WagnerS&P 2017 · 9,786 citations
- Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax ProblemsJunchi Yang, Negar Kiyavash, Niao HeNeurIPS 2020 · 136 citations
- Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max OptimizationYan Yan, Yi Xu, Qihang Lin, Wei Liu et al.NeurIPS 2020 · 70 citations
Related papers
- Amata: An Annealing Mechanism for Adversarial Training AccelerationNanyang Ye, Qianxiao Li, Xiao-Yun Zhou, Zhanxing ZhuAAAI 2021 · 18 citations
- Compositional Training for End-to-End Deep AUC MaximizationZhuoning Yuan, Zhishuai Guo, Nitesh V. Chawla, Tianbao YangICLR 2022 · 34 citations
- Improving Adversarial Robustness by Putting More Regularizations on Less Robust SamplesDongyoon Yang, Insung Kong, Yongdai KimICML 2023 · 15 citations
- Joint-Label Learning by Dual Augmentation for Time Series ClassificationQianli Ma, Zhenjing Zheng, Jiawei Zheng, Sen Li et al.AAAI 2021 · 17 citations
- Functional Virtual Adversarial Training for Semi-Supervised Time Series ClassificationQingyi Pan, Yicheng LiNeurIPS 2025
