Lune

ICLR2024Top-tier venue

Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks

Tianyu Fan, Lirong Wu, Yufei Huang, Haitao Lin, Cheng Tan, Zhangyang Gao, Stan Z. Li

Abstract

Recent years have witnessed the great success of graph pre-training for graph representation learning. With hundreds of graph pre-training tasks proposed, integrating knowledge acquired from multiple pre-training tasks has become a popular research topic. In this paper, we identify two important collaborative processes for this topic: (1) select: how to select an optimal task combination from a given task pool based on their compatibility, and (2) weigh: how to weigh the selected tasks based on their importance. While there currently has been a lot of work focused on weighing, comparatively little effort has been devoted to selecting. This paper proposes a novel instance-level framework for integrating multiple graph pre-training tasks, Weigh And Select (WAS), where the two collaborative processes, weighing and selecting, are combined by decoupled siamese networks. Specifically, it first adaptively learns an optimal combination of tasks for each instance from a given task pool, based on which a customized instance-level task weighing strategy is learned. Extensive experiments on 16 graph datasets across node-level and graph-level downstream tasks have demonstrated that by combining a few simple but classical tasks, WAS can achieve comparable performance to other leading counterparts. The code is available at https://github.com/TianyuFan0504/WAS .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Open Graph Benchmark: Datasets for Machine Learning on Graphs cites Strategies for Pre-training Graph Neural Networks
  • ST-GCond: Self-supervised and Transferable Graph Dataset Condensation cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
  • ST-GCond: Self-supervised and Transferable Graph Dataset Condensation cites Strategies for Pre-training Graph Neural Networks
  • ST-GCond: Self-supervised and Transferable Graph Dataset Condensation cites Graph Contrastive Learning with Augmentations
  • Graph Contrastive Learning with Augmentations cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
  • Graph Contrastive Learning with Augmentations cites Contrastive Multi-View Representation Learning on Graphs
  • Feedback Control for Multi-Objective Graph Self-Supervision cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
  • Feedback Control for Multi-Objective Graph Self-Supervision cites Contrastive Multi-View Representation Learning on Graphs
  • Feedback Control for Multi-Objective Graph Self-Supervision cites Gradient Surgery for Multi-Task Learning
  • Feedback Control for Multi-Objective Graph Self-Supervision cites Graph Contrastive Learning with Augmentations
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. ST-GCond: Self-supervised and Transferable Graph Dataset CondensationBeining Yang, Qingyun Sun, Cheng Ji, Xingcheng Fu et al.ICLR 2025
  2. HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual AdaptersYujie Mo, Runpeng Yu, Xiaofeng Zhu, Xinchao WangICLR 2025
  3. Feedback Control for Multi-Objective Graph Self-SupervisionKarish Grover, Theodore Vasiloudis, Han Xie, Sixing Lu et al.ICML 2026

Builds on29

  1. Open Graph Benchmark: Datasets for Machine Learning on GraphsWeihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong et al.NeurIPS 2020 · 3,935 citations
  2. Graph Contrastive Learning with AugmentationsYuning You, Tianlong Chen, Yongduo Sui, Ting Chen et al.NeurIPS 2020 · 3,042 citations
  3. Gradient Surgery for Multi-Task LearningTianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine et al.NeurIPS 2020 · 2,261 citations
  4. Strategies for Pre-training Graph Neural NetworksWeihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik et al.ICLR 2020 · 1,744 citations
  5. Contrastive Multi-View Representation Learning on GraphsKaveh Hassani, Amir Hosein Khas AhmadiICML 2020 · 1,663 citations

Related papers

  1. Adaptive Transfer Learning on Graph Neural NetworksXueting Han, Zhenhuan Huang, Bang An, Jing BaiKDD 2021 · 30 citations
  2. Better with Less: A Data-Active Perspective on Pre-Training Graph Neural NetworksJiarong Xu, Renhong Huang, Xin Jiang, Yuxuan Cao et al.NeurIPS 2023 · 26 citations
  3. Centrality-guided Pre-training for GraphBin Liang, Shiwei Chen, Lin Gui, Hui Wang et al.ICLR 2025
  4. GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural NetworksZemin Liu, Xingtong Yu, Yuan Fang, Xinming ZhangWWW 2023 · 263 citations
  5. Multi-task Graph Neural Architecture Search with Task-aware Collaboration and CurriculumYijian Qin, Xin Wang, Ziwei Zhang, Hong Chen et al.NeurIPS 2023 · 27 citations
Dusk over the sea between two cliffs drawn in fine vertical lines