A Large-scale Training Paradigm for Graph Generative Models

Large Generative Models (LGMs) such as GPT, Stable Diffusion, Sora, and Suno are trained on a huge amount of texts, images, videos, and audio that are extremely diverse from numerous domains. This large-scale training paradigm on diverse well-curated data enhances the creativity and diversity of the generated content. However, all previous graph-generative models (e.g., GraphRNN, MD-VAE, MoFlow, GDSS, and DiGress) have been trained only on one dataset each time, which cannot replicate the revolutionary success achieved by LGMs in other fields. To remedy this crucial gap, we propose a large-scale training paradigm that uses a large corpus of graphs (over 5000 graphs) from 13 domains, leading to the development of LARGE GRAPH GENERATIVE MODELS (LGGMS). We empirically demonstrate that the pre-trained LGGMs have superior zero-shot generative capability to existing graph generative models. Furthermore, our pre-trained LGGMs can be easily fine-tuned with graphs from target domains and demonstrate even better performance than those directly trained from scratch, behaving as a solid starting point for real-world customization. Inspired by Stable Diffusion, we further equip LGGMs with the Text-to-Graph generation capability, such as providing the description of the network name and domain (i.e., "The power-1138-bus graph represents a network of buses in a power distribution system.") and network statistics (i.e., "The graph has a low average degree, suitable for modeling social media interactions."). This Text-to-Graph capability integrates the extensive world knowledge in the underlying language model, offering users fine-grained control of the generated graphs. We release the code, the model checkpoint, and the datasets at https://github.com/KINDLab-Fly/LGGM . ============================================================================== * DOMAIN: Animal Social Networks * NAME: reptilia-tortoise-network-sl * TEXT: The reptilia-tortoise-network-sl graph represents the social connections among tortoises in the reptile community. ============================================================================== * DOMAIN: Power Networks * NAME: power-eris1176 * TEXT: The power-eris1176 graph represents the interconnected nodes and edges of a power network system ============================================================================== * DOMAIN: Economic Networks * NAME: econ-poli * TEXT: The econ-poli graph represents the interconnectedness of economic and political factors in a network. ============================================================================== * DOMAIN: Ecology Networks * NAME: eco-evergla * TEXT: The eco-evergla graph represents the interconnectedness of species in the Everglades ecosystem. ============================================================================== * DOMAIN: Email Networks * NAME: email-enron-only * TEXT: The email-enron-only graph represents the network of email communication within the Enron corporation. ============================================================================== * DOMAIN: Road Networks * NAME: road-roadNet-CA * TEXT: The road-roadNet-CA graph represents the road network in California. ============================================================================== * DOMAIN: Retweet Networks * NAME: rt_occupywallstnyc * TEXT: The graph rt_occupywallstnyc represents retweet relationships in the Occupy Wall Street movement in New York City. ============================================================================== * DOMAIN: Facebook Networks * NAME: socfb-Haverford76 * TEXT: The socfb-Haverford76 graph represents the social connections among users in the Haverford College community on Facebook. ============================================================================== * DOMAIN: Web Graphs * NAME: web-wiki-chameleon * TEXT: The web-wiki-chameleon graph represents the interconnections between web pages, Wikipedia articles, and chameleon species. ============================================================================== * DOMAIN: Biological Networks * NAME: bio-WormNet-v3-benchmark * TEXT: The bio-WormNet-v3-benchmark graph represents a biological network related to worms. ============================================================================== * DOMAIN: Citation Networks * NAME: cit-DBLP * TEXT: cit-DBLP is a graph representing the citation relationships between research papers in the field of computer science. ============================================================================== * DOMAIN: Collaboration Networks * NAME: ca-netscienc * TEXT: The ca-netscienc graph represents a collaboration network in the field of science. ============================================================================== * You can also sometimes specify a concrete application scenario of the generated network. * Please be accurate but also diverse ============================================================================== PROMPT: Please generate a shor