Utilizing Expert Features for Contrastive Learning of Time-Series Representations
Manuel T. Nonnenmacher, Lukas Oldenburg, Ingo Steinwart, David Reeb
Abstract
We present an approach that incorporates expert knowledge for time-series representation learning. Our method employs expert features to replace the commonly used data transformations in previous contrastive learning approaches. We do this since time-series data frequently stems from the industrial or medical field where expert features are often available from domain experts, while transformations are generally elusive for time-series data. We start by proposing two properties that useful time-series representations should fulfill and show that current representation learning approaches do not ensure these properties. We therefore devise ExpCLR, a novel contrastive learning approach built on an objective that utilizes expert features to encourage both properties for the learned representation. Finally, we demonstrate on three real-world time-series datasets that ExpCLR surpasses several state-of-the-art methods for both unsupervised and semi-supervised representation learning.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers10
- SimMTM: A Simple Pre-Training Framework for Masked Time-Series ModelingJiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang et al.NeurIPS 2023 · 225 citations
- Contrast Everything: A Hierarchical Contrastive Framework for Medical Time-SeriesYihe Wang, Yu Han, Haishuai Wang, Xiang ZhangNeurIPS 2023 · 106 citations
- FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent SpaceShengzhong Liu, Tomoyoshi Kimura, Dongxin Liu, Ruijie Wang et al.NeurIPS 2023 · 72 citations
- Parametric Augmentation for Time Series Contrastive LearningXu Zheng, Tianchun Wang, Wei Cheng, Aitian Ma et al.ICLR 2024 · 29 citations
- Supervised Contrastive Few-Shot Learning for High-Frequency Time SeriesXi Chen, Cheng Ge, Ming Wang, Jin WangAAAI 2023 · 15 citations
Builds on12
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Bootstrap Your Own Latent - A New Approach to Self-Supervised LearningJean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec et al.NeurIPS 2020 · 9,171 citations
- Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 2,360 citations
- Debiased Contrastive LearningChing-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba et al.NeurIPS 2020 · 761 citations
Related papers
- FreRA: A Frequency-Refined Augmentation for Contrastive Learning on Time Series ClassificationTian Tian, Chunyan Miao, Hangwei QianKDD 2025 · 4 citations
- TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation LearningJiexi Liu, Songcan ChenAAAI 2024 · 128 citations
- Soft Contrastive Learning for Time SeriesSeunghan Lee, Taeyoung Park, Kibok LeeICLR 2024 · 63 citations
- TimeDRL: Disentangled Representation Learning for Multivariate Time-SeriesChing Chang, Chiao-Tung Chan, Wei-Yao Wang, Wen-Chih Peng et al.ICDE 2024 · 21 citations
- Temporal-Frequency Co-training for Time Series Semi-supervised LearningZhen Liu, Qianli Ma, Peitian Ma, Linghao WangAAAI 2023 · 39 citations
