Holographic Node Representations: Pre-training Task-Agnostic Node Embeddings
Beatrice Bevilacqua, Joshua Robinson, Jure Leskovec, Bruno Ribeiro
Abstract
Large general purpose pre-trained models have revolutionized computer vision and natural language understanding. However, the development of general purpose pre-trained Graph Neural Networks (GNNs) lags behind other domains due to the lack of suitable generalist node representations. Existing GNN architectures are often tailored to specific task orders, such as node-level, link-level, or higher-order tasks, because different tasks require distinct permutation symmetries, which are difficult to reconcile within a single model. In this paper, we propose holographic node representations, a new blueprint for node representations capable of solving tasks of any order. Holographic node representations have two key components: (1) a task-agnostic expansion map, which produces highly expressive, high-dimensional embeddings, free from node-permutation symmetries, to be fed into (2) a reduction map that carefully reintroduces the relevant permutation symmetries to produce low-dimensional, task-specific embeddings. We show that well-constructed expansion maps enable simple and efficient reduction maps, which can be adapted for any task order. Empirical results show that holographic node representations can be effectively pre-trained and reused across tasks of varying orders, yielding up to 100% relative performance improvement, including in cases where prior methods fail entirely.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Bridging Input Feature Spaces Towards Graph Foundation Models cites Recipe for a General, Powerful, Scalable Graph Transformer
- Bridging Input Feature Spaces Towards Graph Foundation Models cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
- Bridging Input Feature Spaces Towards Graph Foundation Models cites Zero-Shot Generalization of GNNs over Distinct Attribute Domains
- Recipe for a General, Powerful, Scalable Graph Transformer cites Do Transformers Really Perform Badly for Graph Representation?
- Recipe for a General, Powerful, Scalable Graph Transformer cites Rethinking Graph Transformers with Spectral Attention
- Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
- Do Transformers Really Perform Badly for Graph Representation? cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
- Do Transformers Really Perform Badly for Graph Representation? cites Rethinking Graph Transformers with Spectral Attention
- Rethinking Graph Transformers with Spectral Attention cites Open Graph Benchmark: Datasets for Machine Learning on Graphs
Cited by top-tier papers4
- Multi-Domain Riemannian Graph Gluing for Building Graph Foundation ModelsLi Sun, Zhenhao Huang, Silei Chen, Lanxu Yang et al.ICLR 2026 · 5 citations
- Bridging Input Feature Spaces Towards Graph Foundation ModelsMoshe Eliasof, Krishna Sri Ipsit Mantri, Beatrice Bevilacqua, Bruno Ribeiro et al.ICLR 2026 · 4 citations
- Any-Subgroup Equivariant Networks via Symmetry BreakingAbhinav Goel, Derek Lim, Hannah Lawrence, Stefanie Jegelka et al.ICLR 2026
- Zero-Shot Generalization of GNNs over Distinct Attribute DomainsYangyi Shen, Jincheng Zhou, Beatrice Bevilacqua, Joshua Robinson et al.ICML 2025
Builds on41
- Open Graph Benchmark: Datasets for Machine Learning on GraphsWeihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong et al.NeurIPS 2020 · 3,935 citations
- Do Transformers Really Perform Badly for Graph Representation?Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng et al.NeurIPS 2021 · 1,632 citations
- Recipe for a General, Powerful, Scalable Graph TransformerLadislav Rampásek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu et al.NeurIPS 2022 · 1,216 citations
- Rethinking Graph Transformers with Spectral AttentionDevin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau et al.NeurIPS 2021 · 854 citations
- Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link PredictionZhaocheng Zhu, Zuobai Zhang, Louis-Pascal A. C. Xhonneux, Jian TangNeurIPS 2021 · 546 citations
Related papers
- Strategies for Pre-training Graph Neural NetworksWeihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik et al.ICLR 2020 · 1,744 citations
- MORE: Molecule Pretraining with Multi-Level Pretext TaskYeongyeong Son, Dasom Noh, Gyoungyoung Heo, Gyoung Jin Park et al.AAAI 2025 · 1 citation
- Instruction-based Hypergraph PretrainingMingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu et al.SIGIR 2024 · 4 citations
- Train Once and Explain Everywhere: Pre-training Interpretable Graph Neural NetworksJun Yin, Chaozhuo Li, Hao Yan, Jianxun Lian et al.NeurIPS 2023 · 16 citations
- Equivariance Everywhere All At Once: A Recipe for Graph Foundation ModelsBen Finkelshtein, Ismail Ilkan Ceylan, Michael M. Bronstein, Ron LevieNeurIPS 2025 · 21 citations
