Multimodal Graph Representation Learning with Dynamic Information Pathways
Xiaobin Hong, Mingkai Lin, Xiaoli Wang, Chaoqun Wang, Wenzhong Li
摘要
Multimodal graphs, where nodes contain heterogeneous features such as images and text, are increasingly common in real-world applications. Effectively learning on such graphs requires both adaptive intra-modal message passing and efficient inter-modal aggregation. However, most existing approaches to multimodal graph learning are typically extended from conventional graph neural networks and rely on static structures or dense attention, which limit flexibility and expressive node embedding learning. In this paper, we propose a novel multimodal graph representation learning framework with Dynamic information Pathways (DiP). By introducing modality-specific pseudo nodes, DiP enables dynamic message routing within each modality via proximity-guided pseudo-node interactions and captures inter-modality dependence through efficient information pathways in a shared state space. This design achieves adaptive, expressive, and sparse message propagation across modalities with linear complexity. We conduct the link prediction and node classification tasks to evaluate performance and carry out full experimental analyses. Extensive experiments across multiple benchmarks demonstrate that DiP consistently outperforms baselines.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper25
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- Open Graph Benchmark: Datasets for Machine Learning on GraphsWeihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong 等NeurIPS 2020 · 被引用 3,935 次
- Graph Neural Networks Exponentially Lose Expressive Power for Node ClassificationKenta Oono, Taiji SuzukiICLR 2020 · 被引用 864 次
- Understanding over-squashing and bottlenecks on graphs via curvatureJake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong 等ICLR 2022 · 被引用 628 次
相关 Paper
- Towards Dynamic Message Passing on GraphsJunshu Sun, Chenxue Yang, Xiangyang Ji, Qingming Huang 等NeurIPS 2024 · 被引用 19 次
- DySarl: Dynamic Structure-Aware Representation Learning for Multimodal Knowledge Graph ReasoningKangzheng Liu, Feng Zhao, Yu Yang, Guandong XuACM MM 2024 · 被引用 4 次
- D2R: Dual-Branch Dynamic Routing Network for Multimodal Sentiment DetectionYifan Chen, Kuntao Li, Weixing Mai, Qiaofeng Wu 等EMNLP 2024 · 被引用 10 次
- Dynamic Embedding on Textual Networks via a Gaussian ProcessPengyu Cheng, Yitong Li, Xinyuan Zhang, Liqun Chen 等AAAI 2020 · 被引用 10 次
- DIGEST: Dynamic Graph Refinement with Dual Contrastive Semantic Transfer for Multimodal RecommendationXiangyu Sai, Meysam Madadi, Sergio Escalera, Yong XuSIGIR 2026
