BERT-Trip: Effective and Scalable Trip Representation using Attentive Contrast Learning

Trip recommendation has drawn considerable attention over the past decade. In trip recommendation, a sequence of point-of-interests (POIs) are recommended for a given query which includes an origin and a destination. Recently the emergence of the attention mechanism and many attention-incorporated models have achieved great success in various fields. Trip recommendation problems demonstrate similar characteristics that can potentially benefit from the attention mechanism. However, applying the attention mechanism for trip recommendation is non-trivial. We are motivated to answer the following two research questions. (1) How can we learn trip representation effectively without labels? Unlike most of the natural language processing tasks, there are no ground-truth labels available for trip recommendation. (2) How can we learn trip representation effectively without handcrafting negative samples? In this paper, we cast the trip representation learning into a natural language processing (NLP) task. We propose BERT-Trip, a self-supervised contrast learning framework, to learn effective and scalable trip representation in support of time-sensitive and user-personalized trip recommendation. BERT-Trip builds on a Siamese network to maximize the similarity between the augmentations of trips with BERT as the backbone encoder. We utilize the masking strategy for generating augmented views (positive sample pairs) of trips in the Siamese network and employ the stop-gradient on one side of the Siamese network to eliminate the need to use any negative sample pairs or momentum encoders. Extensive experiments on real-world datasets demonstrate that BERT-Trip consistently outperformed the state-of-the-art methods in terms of all effectiveness metrics. Compared with the state-of-the-art methods, BERT-Trip is able to yield up to 24 percent and 40 percent increases in F1score on the Flickr and the Weeplaces datasets, respectively. A rigorous performance evaluation of BERT-Trip on scalability up to 12800 POIs is also provided.