Scalable Feature Learning on Huge Knowledge Graphs for Downstream Machine Learning

Many machine learning tasks can benefit from external knowledge. Large knowledge graphs store such knowledge, and embedding methods can be used to distill it into ready-to-use vector representations for downstream applications. For this purpose, current models have however two limitations: they are primarily optimized for link prediction, via local contrastive learning, and their application to the largest graphs requires significant engineering effort due to GPU memory limits. To address these, we introduce SEPAL: a Scalable Embedding Propagation ALgorithm for large knowledge graphs designed to produce high-quality embeddings for downstream tasks at scale. The key idea of SEPAL is to ensure global embedding consistency by optimizing embeddings only on a small core of entities, and then propagating them to the rest of the graph with message passing. We evaluate SEPAL on 7 large-scale knowledge graphs and 46 downstream machine learning tasks. Our results show that SEPAL significantly outperforms previous methods on downstream tasks. In addition, SEPAL scales up its base embedding model, enabling fitting huge knowledge graphs on commodity hardware. Our code is available at: https://github.com/soda-inria/sepal . 1 Introduction: embedding knowledge for downstream tasks External knowledge for machine learning Bringing general knowledge to a machine-learning task revives an old promise of making it easier via this knowledge [Lenat and Feigenbaum, 2000] . Indeed, data science is often about entities of the world-persons, places, organizations-that are well characterized in general-purpose knowledge graphs. These graphs carry rich information, including numerical attributes and relationships between entities, and can be connected to string values in tabular data through entity linking techniques [Mendes et al., 2011 , Foppiano and Romary, 2020 , Delpeuch, 2019] . A thorny challenge, however, is to transform this relational information into features for downstream tabular machine learning [Kanter and Veeramachaneni, 2015 , Cappuzzo et al., 2025 , Robinson et al., 2024] . To that end, a scalable solution is offered by graph embedding methods that distill the graph information into node features readily usable by any downstream tabular learner [