Parameter-free Dynamic Graph Embedding for Link Prediction

Dynamic interaction graphs have been widely adopted to model the evolution of user-item interactions over time. There are two crucial factors when modelling user preferences for link prediction in dynamic interaction graphs: 1) collaborative relationship among users and 2) user personalized interaction patterns. Existing methods often implicitly consider these two factors together, which may lead to noisy user modelling when the two factors diverge. In addition, they usually require time-consuming parameter learning with back-propagation, which is prohibitive for real-time user preference modelling. To this end, this paper proposes FreeGEM, a parameter-free dynamic graph embedding method for link prediction. Firstly, to take advantage of the collaborative relationships, we propose an incremental graph embedding engine to obtain user/item embeddings, which is an Online-Monitor-Offline architecture consisting of an Online module to approximately embed users/items over time, a Monitor module to estimate the approximation error in real time and an Offline module to calibrate the user/item embeddings when the online approximation errors exceed a threshold. Meanwhile, we integrate attribute information into the model, which enables FreeGEM to better model users belonging to some under represented groups. Secondly, we design a personalized dynamic interaction pattern modeller, which combines dynamic time decay with attention mechanism to model user short-term interests. Experimental results on two link prediction tasks show that FreeGEM can outperform the state-of-the-art methods in accuracy while achieving over 36X improvement in efficiency. All code and datasets can be found in https://github.com/FudanCISL/FreeGEM . In this paper, we propose a Parameter-Free Dynamic Graph EMbedding (FreeGEM) method for link prediction. Here, parameter-free means that we do not incorporate any parameters which need to be learned via back-propagation. As shown in Figure 1 , FreeGEM consists of two key components: incremental graph embedding engine and personalized dynamic interaction pattern modeller. The incremental graph embedding engine takes both historical data and online stream data as input and generates user/item embeddings as the output, which exploits the collaborative relationship among users. Its core innovation is the proposed Online-Monitor-Offline architecture, which can achieve online embedding updates by solving closed-form solutions in real time and keep the approximation errors caused by online singular value decomposition (SVD) [2] within any predefined threshold. In the Offline step, we propose a frequency-aware preference matrix reconstruction method to alleviate the oversmoothing problem and an attribute-integrated SVD to alleviate the cold-start issue. Surprisingly, the integration of attribute information enables FreeGEM to better model users belonging to under represented groups. In the Online step, we convert the offline truncated SVD into an online SVD to generate embeddings in real time. In the Monitor module, we estimate the online approximation error in real time by analyzing the relationship between approximation error and the update of online algorithm. The personalized dynamic interaction pattern modeller is also a parameter-free component, which combines the dynamic time decay with attention mechanism to model user short-term interests. It takes the user and item embeddings as input and outputs the prediction results. Specifically, it selectively forgets the early interactions through dynamic time decay and hence focuses on more recent interactions for prediction. Then, it leverages the attention mechanism to capture user personalized dynamic interaction patterns over the "decayed" item embedding sequences. Experimental results on two link prediction tasks (future item recommendation and next interaction prediction) show that FreeGEM can substantially outperform the state-of-the-art link prediction methods in accuracy while achieving over 36X improvement in computational efficiency. Besides, our empirical studies also confirm that FreeGEM can alleviate the cold-start issue and achieve high robustness on very sparse data. Related Work Link prediction methods on dynamic interaction graphs are mostly based on TPP, RNN and GNN, etc., which need time-consuming training process. To the best of our knowledge, the proposed FreeGEM is the only parameter-free method in this task, with high accuracy and efficiency. TPP-based methods Know-Evolve [28] and HTNE [41] model interactions as multivariate point processes and Hawkes processes, respectively. Wang et al. [30] propose a co-evolutionary process to model the co-evolving nature of users and items. Shchur et al. [22] propose to directly model the conditional distribution of inter-event times. DSPP [3] incorporates topology and long-term dependencies into the intensity function. RNN-based methods RRN [32] models user and item interaction sequences with sepa