Dual Mamba for Node-Specific Representation Learning: Tackling Over-Smoothing with Selective State Space Modeling
Xin He, Yili Wang, Yiwei Dai, Xin Wang
Abstract
Over-smoothing remains a fundamental challenge in deep Graph Neural Networks (GNNs), where repeated message passing causes node representations to become indistinguishable. While existing solutions, such as residual connections and skip layers, alleviate this issue to some extent, they fail to explicitly model how node representations evolve in a node-specific and progressive manner across layers. Moreover, these methods do not take global information into account, which is also crucial for mitigating the over-smoothing problem. To address the aforementioned issues, in this work, we propose a Dual Mamba-enhanced Graph Convolutional Network (DMbaGCN), which is a novel framework that integrates Mamba into GNNs to address over-smoothing from both local and global perspectives. DMbaGCN consists of two modules: the Local State-Evolution Mamba (LSEMba) for local neighborhood aggregation and utilizing Mamba’s selective state space modeling to capture node-specific representation dynamics across layers, and the Global Context-Aware Mamba (GCAMba) that leverages Mamba’s global attention capabilities to incorporate global context for each node. By combining these components, DMbaGCN enhances node discriminability in deep GNNs, thereby mitigating over-smoothing. Extensive experiments on multiple benchmarks demonstrate the effectiveness and efficiency of our method.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on23
- Simple and Deep Graph Convolutional NetworksMing Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding et al.ICML 2020 · 1,910 citations
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space DualityTri Dao, Albert GuICML 2024 · 1,407 citations
- Recipe for a General, Powerful, Scalable Graph TransformerLadislav Rampásek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu et al.NeurIPS 2022 · 1,216 citations
- HiPPO: Recurrent Memory with Optimal Polynomial ProjectionsAlbert Gu, Tri Dao, Stefano Ermon, Atri Rudra et al.NeurIPS 2020 · 1,100 citations
- Rethinking Graph Transformers with Spectral AttentionDevin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau et al.NeurIPS 2021 · 854 citations
Related papers
- HSA-Net: Hierarchical and Structure-Aware Framework for Efficient and Scalable Molecular Language ModelingZihang Shao, Wentao Lei, Lei Wang, Wencai Ye et al.AAAI 2026
- DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning with Selective State Space ModelsHaonan Yuan, Qingyun Sun, Zhaonan Wang, Xingcheng Fu et al.AAAI 2025 · 14 citations
- MultiNet: Adaptive Multi-Viewed Subgraph Convolutional Networks for Graph ClassificationXinya Qin, Lu Bai, Lixin Cui, Ming Li et al.NeurIPS 2025 · 2 citations
- Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptative Residual ModuleJingbo Zhou, Yixuan Du, Ruqiong Zhang, Jun Xia et al.NeurIPS 2024 · 6 citations
- Towards Deeper Graph Neural Networks with Differentiable Group NormalizationKaixiong Zhou, Xiao Huang, Yuening Li, Daochen Zha et al.NeurIPS 2020 · 248 citations
