FAT: Frequency-Aware Pretraining for Enhanced Time-Series Representation Learning

Recent advancements in time-series forecasting have highlighted the importance of frequency-domain modeling. However, deep learning models primarily operate in the time domain, limiting their ability to capture frequency-based patterns. Existing approaches normally introduce novel neural network architectures tailored to task-specific frequency properties, yet they often lack generalization and require extensive domain-specific adaptations. In this paper, we propose FAT, a novel pretraining framework that learns generalizable Frequency-Aware Time-series representations through self-supervised learning. The key idea of FAT is to pretrain any backbone model to directly extract generalizable frequency patterns from time-domain signals and encode them into robust representations-eliminating the need for architectural modifications or additional modules during inference. This is achieved via a frequency reformer that amplifies critical frequency components learned through self-supervision and enforces similarity between the original and frequency-reformed time-series representations produced by the encoder. In addition, recognizing that semantically equivalent time-series can exhibit different frequency expressions-analogous to how the same phrase is pronounced differently by different speakers-FAT introduces a Knowledge-Guided Frequency Reformer that unifies the expression of frequency patterns with the same underlying semantics and extends similarity constraints to frequency-invariant augmented samples to enhance robustness of learned representation. Experiments on 14 benchmark datasets across regression and classification tasks show that FAT consistently achieves state-of-the-art performance while maintaining robustness across diverse backbone models, significantly outperforming existing pretraining methods. Our code is available at https://github.com/JiaXiangfei/FAT.