Rethinking Structural Encodings: Adaptive Graph Transformer for Node Classification Task

Graph Transformers have proved their advantages in graph data mining with elaborate Positional Encodings, especially in graph-level tasks. However, their application in the node classification task has not been fully exploited yet. In the node classification task, existing Graph Transformers with Positional Encodings are limited by the following issues: (i) PEs describing the node’s positional identities are insufficient for the node classification task on complex graphs, where a full portrayal of the local node property is needed. (ii) PEs for graphs are integrated with Transformers in a constant schema, resulting in the ignorance of local patterns that may vary among different nodes. In this paper, we propose Adaptive Graph Transformer (AGT) to tackle above issues. AGT consists of a Learnable Centrality Encoding and a Kernelized Local Structure Encoding. The two modules extract structural patterns from centrality and subgraph views in a learnable and scalable manner. Further, we design the Adaptive Transformer Block to adaptively integrate the attention scores and Structural Encodings in a node-specific manner. AGT achieves state-of-the-art performances on nine real-world web graphs (up to 1.6 million nodes). Furthermore, AGT shows outstanding results on two series of synthetic graphs with ranges of heterophily and noise ratios.