Revisiting Positive Samples in Graph Contrastive Learning: From the Perspective of Message Passing
Lianze Shan, Ningchong Wang, Jitao Zhao, Di Jin, Dongxiao He
摘要
Graph Contrastive Learning (GCL), which trains graph encoders by maximizing similarity between positive samples and minimizing it between negative ones, has emerged as a mainstream graph pre-training paradigm. It is widely recognized that positive samples are essential in GCLs. Ideally, maximizing the similarity of positive samples enables graph encoders to capture intrinsic semantics and patterns of graph data. However, we discover an interesting phenomenon: GCLs can achieve competitive performance even without positive samples. This motivates us to revisit the fundamental mechanism of positive samples in GCLs. From the perspective of Dirichlet energy, we theoretically find that message passing, a key mechanism in graph encoders, trivializes the maximization of positive samples, preventing GCLs from effectively learning from positive samples. To address this, we propose SPGCL to mitigate the trivialization caused by message passing and restore the learning efficacy of positive samples. Specifically, we find that high Dirichlet energy features help positive samples provide effective learning signals while low Dirichlet energy features contribute little to positive learning signal but is useful for positive sampling. Based on this, SPGCL introduces an energy-aware propagation mechanism that selectively propagates features based on their Dirichlet energy, preventing low-energy features from dominating similarity. Furthermore, we construct a positive sampling matrix based on Dirichlet energy to exclude misleading positive samples and focus on node pairs with higher discrimination. Extensive experiments demonstrate the effectiveness of SPGCL.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper19
- LightGCN: Simplifying and Powering Graph Convolution Network for RecommendationXiangnan He, Kuan Deng, Xiang Wang, Yan Li 等SIGIR 2020 · 被引用 4,448 次
- Geom-GCN: Geometric Graph Convolutional NetworksHongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei 等ICLR 2020 · 被引用 1,445 次
- Graph Contrastive Learning with Adaptive AugmentationYanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu 等WWW 2021 · 被引用 1,415 次
- Adversarial Graph Augmentation to Improve Graph Contrastive LearningSusheel Suresh, Pan Li, Cong Hao, Jennifer NevilleNeurIPS 2021 · 被引用 475 次
- Augmentation-Free Self-Supervised Learning on GraphsNamkyeong Lee, Junseok Lee, Chanyoung ParkAAAI 2022 · 被引用 288 次
相关 Paper
- Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive LearningXiaojun Guo, Yifei Wang, Zeming Wei, Yisen WangNeurIPS 2023 · 被引用 20 次
- GradGCL: Gradient Graph Contrastive LearningRan Li, Shimin Di, Lei Chen, Xiaofang ZhouICDE 2024 · 被引用 3 次
- ReGCL: Rethinking Message Passing in Graph Contrastive LearningCheng Ji, Zixuan Huang, Qingyun Sun, Hao Peng 等AAAI 2024 · 被引用 23 次
- ProGCL: Rethinking Hard Negative Mining in Graph Contrastive LearningJun Xia, Lirong Wu, Ge Wang, Jintao Chen 等ICML 2022 · 被引用 174 次
- InfoNCE is a Free Lunch for Semantically guided Graph Contrastive LearningZixu Wang, Bingbing Xu, Yige Yuan, Huawei Shen 等SIGIR 2025 · 被引用 4 次
