Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long
Abstract
Unsupervised detection of anomaly points in time series is a challenging problem, which requires the model to derive a distinguishable criterion. Previous methods tackle the problem mainly through learning pointwise representation or pairwise association, however, neither is sufficient to reason about the intricate dynamics. Recently, Transformers have shown great power in unified modeling of pointwise representation and pairwise association, and we find that the self-attention weight distribution of each time point can embody rich association with the whole series. Our key observation is that due to the rarity of anomalies, it is extremely difficult to build nontrivial associations from abnormal points to the whole series, thereby, the anomalies' associations shall mainly concentrate on their adjacent time points. This adjacent-concentration bias implies an association-based criterion inherently distinguishable between normal and abnormal points, which we highlight through the Association Discrepancy. Technically, we propose the Anomaly Transformer with a new Anomaly-Attention mechanism to compute the association discrepancy. A minimax strategy is devised to amplify the normal-abnormal distinguishability of the association discrepancy. The Anomaly Transformer achieves state-of-the-art results on six unsupervised time series anomaly detection benchmarks of three applications: service monitoring, space&earth exploration, and water treatment.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
128 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting cites Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
- Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting cites Language Models are Few-Shot Learners
- ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
- ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis cites Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
- ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis cites TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
- ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale cites Language Models are Few-Shot Learners
- TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
- TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis cites Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
- TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis cites Language Models are Few-Shot Learners
- FITS: Modeling Time Series with 10k Parameters cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
- FITS: Modeling Time Series with 10k Parameters cites Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
- FITS: Modeling Time Series with 10k Parameters cites TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- MOMENT: A Family of Open Time-series Foundation Models cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
- MOMENT: A Family of Open Time-series Foundation Models cites Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
- MOMENT: A Family of Open Time-series Foundation Models cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- MOMENT: A Family of Open Time-series Foundation Models cites TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
- MOMENT: A Family of Open Time-series Foundation Models cites One Fits All: Power General Time Series Analysis by Pretrained LM
- One Fits All: Power General Time Series Analysis by Pretrained LM cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
- One Fits All: Power General Time Series Analysis by Pretrained LM cites Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
- One Fits All: Power General Time Series Analysis by Pretrained LM cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- One Fits All: Power General Time Series Analysis by Pretrained LM cites TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
- One Fits All: Power General Time Series Analysis by Pretrained LM cites Language Models are Few-Shot Learners
Cited by top-tier papers128
- One Fits All: Power General Time Series Analysis by Pretrained LMTian Zhou, Peisong Niu, Xue Wang, Liang Sun et al.NeurIPS 2023 · 1,178 citations
- MOMENT: A Family of Open Time-series Foundation ModelsMononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai et al.ICML 2024 · 442 citations
- TimesNet: Temporal 2D-Variation Modeling for General Time Series AnalysisHaixu Wu, Tengge Hu, Yong Liu, Hang Zhou et al.ICLR 2023 · 423 citations
- ModernTCN: A Modern Pure Convolution Structure for General Time Series AnalysisDonghao Luo, Xue WangICLR 2024 · 392 citations
- FITS: Modeling Time Series with 10k ParametersZhijian Xu, Ailing Zeng, Qiang XuICLR 2024 · 259 citations
Builds on10
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Informer: Beyond Efficient Transformer for Long Sequence Time-Series ForecastingHaoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang et al.AAAI 2021 · 7,289 citations
- Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series ForecastingHaixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng LongNeurIPS 2021 · 5,824 citations
Related papers
- Low Rank Transformer for Multivariate Time Series Anomaly Detection and LocalizationCharalampos Shimillas, Kleanthis Malialis, Konstantinos Fokianos, Marios M. PolycarpouICLR 2026
- Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly DetectionXincheng Yao, Ruoqi Li, Zefeng Qian, Yan Luo et al.ICCV 2023 · 48 citations
- SARAD: Spatial Association-Aware Anomaly Detection and Diagnosis for Multivariate Time SeriesZhihao Dai, Ligang He, Shuanghua Yang, Matthew LeekeNeurIPS 2024 · 27 citations
- When Model Meets New Normals: Test-Time Adaptation for Unsupervised Time-Series Anomaly DetectionDongmin Kim, Sunghyun Park, Jaegul ChooAAAI 2024 · 43 citations
- HYDRA: A Multi-Level Hierarchy-Driven Approach for Robust Anomaly Detection in Time SeriesMingyi Huang, Qinghua Liu, Paul Boniol, John PaparrizosSIGMOD 2026 · 4 citations
