SGNN: Efficient Global Mixing and Local Message Passing for Long-Range Graph Learning
Dai Shi, Linhan Luo, Luke Thompson, Lequan Lin, Andi Han, Junbin Gao, Jose Miguel Hernandez-Lobato
Abstract
Message-passing neural networks (MPNNs) often suffer from an information bottleneck when capturing long-range dependencies, leading to the oversquashing (OSQ) phenomenon. Alongside spatial connectivity enrichment (e.g., rewiring), recent studies have shown that spectral filtering can yield strong long-range learning outcomes, as spectral operators enable global information mixing that alleviates OSQ. These approaches achieve this either by stabilizing the Jacobian energies in deep propagation or by guaranteeing OSQ mitigation under strong theoretical assumptions. We revisit these conclusions and show that the associated Jacobian sensitivity lower bound is generally difficult to achieve in practice. We then propose SGNN, which mitigates OSQ without such restrictive assumptions by lightweightly reintroducing omitted components with substantially lower computational complexity, while standard stability constraints on feature transformations remain effective under our new dynamics. Extensive experiments across diverse domains (e.g., long-range benchmarks, KGQA, and mesh-based fluid dynamics) demonstrate that SGNN achieves up to an order-of-magnitude error reduction with up to 50% fewer parameters. Our code can be found in https://anonymous.4open.science/r/S3GNN-2359.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on29
- Do Transformers Really Perform Badly for Graph Representation?Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng et al.NeurIPS 2021 · 1,632 citations
- Recipe for a General, Powerful, Scalable Graph TransformerLadislav Rampásek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu et al.NeurIPS 2022 · 1,216 citations
- Learning Mesh-Based Simulation with Graph NetworksTobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, Peter W. BattagliaICLR 2021 · 1,175 citations
- Principal Neighbourhood Aggregation for Graph NetsGabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò et al.NeurIPS 2020 · 914 citations
- Rethinking Graph Transformers with Spectral AttentionDevin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau et al.NeurIPS 2021 · 854 citations
Related papers
- Locality-Aware Graph Rewiring in GNNsFederico Barbero, Ameya Velingker, Amin Saberi, Michael M. Bronstein et al.ICLR 2024 · 64 citations
- Effects of Dropout on Performance in Long-range Graph Learning TasksJasraj Singh, Keyue Jiang, Brooks Paige, Laura ToniNeurIPS 2025 · 2 citations
- Spatio-Spectral Graph Neural NetworksSimon Geisler, Arthur Kosmala, Daniel Herbst, Stephan GünnemannNeurIPS 2024 · 37 citations
- Deeper with Riemannian Geometry: Overcoming Oversmoothing and Oversquashing for Graph Foundation ModelsLi Sun, Zhenhao Huang, Ming Zhang, Philip S. YuNeurIPS 2025 · 10 citations
- Mitigating Over-Squashing in Graph Neural Networks by Spectrum-Preserving SparsificationLangzhang Liang, Fanchen Bu, Zixing Song, Zenglin Xu et al.ICML 2025
