RLPer: A Reinforcement Learning Model for Personalized Search
Abstract
Personalized search improves generic ranking models by taking user interests into consideration and returning more accurate search results to individual users. In recent years, machine learning and deep learning techniques have been successfully applied in personalized search. Most existing personalization models simply regard the search history as a static set of user behaviours and learn fixed ranking strategies based on the recorded data. Though improvements have been observed, it is obvious that these methods ignore the dynamic nature of the search process: search is a sequence of interactions between the search engine and the user. During the search process, the user interests may dynamically change. It would be more helpful if a personalized search model could track the whole interaction process and update its ranking strategy continuously. In this paper, we propose a reinforcement learning based personalization model, referred to as RLPer, to track the sequential interactions between the users and search engine with a hierarchical Markov Decision Process (MDP). In RLPer, the search engine interacts with the user to update the underlying ranking model continuously with real-time feedback. And we design a feedback-aware personalized ranking component to catch the user’s feedback which has impacts on the user interest profile for the next query. Experimental results on the publicly available AOL search log verify that our proposed model can significantly outperform state-of-the-art personalized search models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers6
- Group based Personalized Search by Integrating Search Behaviour and Friend NetworkYujia Zhou, Zhicheng Dou, Bingzheng Wei, Ruobing Xie et al.SIGIR 2021 · 29 citations
- IHGNN: Interactive Hypergraph Neural Network for Personalized Product SearchDian Cheng, Jiawei Chen, Wenjun Peng, Wenqin Ye et al.WWW 2022 · 25 citations
- FedPS: A Privacy Protection Enhanced Personalized Search FrameworkJing Yao, Zhicheng Dou, Ji-Rong WenWWW 2021 · 11 citations
- Incorporating Explicit Subtopics in Personalized SearchShuting Wang, Zhicheng Dou, Jing Yao, Yujia Zhou et al.WWW 2023 · 10 citations
- MA4DIV: Multi-Agent Reinforcement Learning for Search Result DiversificationYiqun Chen, Jiaxin Mao, Yi Zhang, Dehong Ma et al.WWW 2025 · 7 citations
Related papers
- Long Short-Term Session Search: Joint Personalized Reranking and Next Query PredictionQiannan Cheng, Zhaochun Ren, Yujie Lin, Pengjie Ren et al.WWW 2021 · 19 citations
- Learning Dynamic Multi-attribute Interest for Personalized Product SearchYutong Bai, Zhicheng Dou, Ji-Rong WenEMNLP Findings 2024
- Cognitive Personalized Search Integrating Large Language Models with an Efficient Memory MechanismYujia Zhou, Qiannan Zhu, Jiajie Jin, Zhicheng DouWWW 2024 · 41 citations
- Match Plan Generation in Web Search with Parameterized Action Reinforcement LearningZiyan Luo, Linfeng Zhao, Wei Cheng, Sihao Chen et al.WWW 2021 · 4 citations
- Orion: Steering Personalized Web Agents via Global-Micro Profiling and Adaptive Intent TrackingDie Hu, Jingguo Ge, Weitao Tang, He Kong et al.AAAI 2026
