Temporal-Series-Aware Adaptive Positional Encoding for Transformer-based Sequential Recommendation

With the rapid proliferation of short-video platforms and content-driven social networks, sequential recommendation models capable of accurately capturing user interests have become increasingly crucial. Among these, Transformer-based sequential recommendation models have gained widespread adoption due to their superior ability. The positional encoding (PE) in Transformer architectures serves to incorporate positional information into sequences. However, relying solely on original absolute positional information may be insufficient for sequential recommendation models. In contrast, the dwell time after interactions (i.e., the time intervals between consecutive user interactions) provides a more accurate reflection of users' emotional responses and evolving interests. Despite its significance, this aspect has often been overlooked in existing works. To fully utilize this information, our work introduces an adaptive PE method, termed TSAPE (Temporal-Series-Aware Positional Encoding). This approach introduces an innovative modeling of the sequence of time intervals between user interactions, rather than the numerical values of the intervals themselves, thereby capturing real-time feedback on user interests and integrating it with conventional PE mechanisms. Furthermore, we employ multiple layers of one-dimensional convolutional networks and attention mechanisms to endow the features with adaptive capabilities across various time interval scenarios. This enables TSAPE to more accurately capture sequential positional information at any given moment. By enhancing the sequential order information of interactions, TSAPE significantly improves the accuracy of next-item recommendations. We seamlessly integrated our method into several Transformer-based sequential recommendation models and conducted comparisons with state-of-the-art sequential recommendation approaches and widely-used PE methods. The results demonstrate that the integration of TSAPE consistently outperforms the original backbone models and other SOTA methods. The SASRec model integrated with TSAPE achieves an average improvement of 15.61% across three evaluation metrics on four benchmark datasets. Our code has been made publicly available at https://github.com/rongbo-qi/TSAPE_Rec.