CoLES: Contrastive Learning for Event Sequences with Self-Supervision
Dmitrii Babaev, Nikita Ovsov, Ivan Kireev, Mariya Ivanova, Gleb Gusev, Ivan Nazarov, Alexander Tuzhilin
摘要
We address the problem of self-supervised learning on discrete event sequences generated by real-world users. Self-supervised learning incorporates complex information from the raw data in low-dimensional fixed-length vector representations that could be easily applied in various downstream machine learning tasks. In this paper, we propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains, to the discrete event sequences domain in a self-supervised setting.
We deployed CoLES embeddings based on sequences of transactions at the large European financial services company. Usage of CoLES embeddings significantly improves the performance of the pre-existing models on downstream tasks and produces significant financial gains, measured in hundreds of millions of dollars yearly. We also evaluated CoLES on several public event sequences datasets and showed that CoLES representations consistently outperform other methods on different downstream tasks.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
已有 2 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。
脉络
- A Simple Framework for Contrastive Learning of Visual Representations 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
- Momentum Contrast for Unsupervised Visual Representation Learning 引用了 A Simple Framework for Contrastive Learning of Visual Representations
- Disentangled Self-Supervision in Sequential Recommenders 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
- GradGCL: Gradient Graph Contrastive Learning 引用了 A Simple Framework for Contrastive Learning of Visual Representations
- LightPath: Lightweight and Scalable Path Representation Learning 引用了 Momentum Contrast for Unsupervised Visual Representation Learning
引用它的顶会 Paper2
它引用的 Paper5
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 次引用
- ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel 等ICLR 2020 · 7,418 次引用
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 次引用
- Disentangled Self-Supervision in Sequential RecommendersJianxin Ma, Chang Zhou, Hongxia Yang, Peng Cui 等KDD 2020 · 223 次引用
- Momentum Contrast for Unsupervised Visual Representation LearningKaiming He, Haoqi Fan, Yuxin Wu, Saining Xie 等CVPR 2020
相关 Paper
- SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence RepresentationsHooman Sedghamiz, Shivam Raval, Enrico Santus, Tuka Alhanai 等EMNLP Findings 2021 · 19 次引用
- REBAR: Retrieval-Based Reconstruction for Time-series Contrastive LearningMaxwell A. Xu, Alexander Moreno, Hui Wei, Benjamin M. Marlin 等ICLR 2024 · 23 次引用
- Contrastive Learning for Sequential RecommendationXu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu 等ICDE 2022 · 674 次引用
- Disentangling Long and Short-Term Interests for RecommendationYu Zheng, Chen Gao, Jianxin Chang, Yanan Niu 等WWW 2022 · 128 次引用
- Intent Contrastive Learning for Sequential RecommendationYongjun Chen, Zhiwei Liu, Jia Li, Julian J. McAuley 等WWW 2022 · 429 次引用
