EMNLP Findings2022
ExpertPLM: Pre-training Expert Representation for Expert Finding
Abstract
Expert Finding is an important task in Community Question Answering (CQA) platforms, which could help route questions to potential users to answer. The key is to learn representations of experts based on their historical answered questions accurately. In this paper, inspired by the strong text understanding ability of Pretrained Language modelings (PLMs), we propose a pre-training and fine-tuning expert finding framework. The core is that we design an expert-level pre-training paradigm, that effectively integrates expert interest and expertise simultaneously. Specifically different from the typical corpus-level pre-training, we treat each expert as the basic pre-training unit including all the historical answered question titles of the expert, which could fully indicate the expert interests for questions. Besides, we integrate the vote score information along with each answer of the expert into the pre-training phrase to model the expert ability explicitly. Finally, we propose a novel reputation-augmented Masked Language Model (MLM) pre-training strategy to capture the expert reputation information. In this way, our method could learn expert representation comprehensively, which then will be adopted and fine-tuned in the down-streaming expert-finding task. Extensive experimental results on six real-world CQA datasets demonstrate the effectiveness of our method.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on2
- U-BERT: Pre-training User Representations for Improved RecommendationZhaopeng Qiu, Xian Wu, Jingyue Gao, Wei FanAAAI 2021 · 171 citations
- PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervisionChuhan Wu, Fangzhao Wu, Tao Qi, Jianxun Lian et al.EMNLP Findings 2020 · 36 citations
Related papers
- Contrastive Pre-training for Personalized Expert FindingQiyao Peng, Hongtao Liu, Zhepeng Lv, Qing Yang et al.EMNLP Findings 2023 · 2 citations
- Towards a Multi-View Attentive Matching for Personalized Expert FindingQiyao Peng, Hongtao Liu, Yinghui Wang, Hongyan Xu et al.WWW 2022 · 20 citations
- Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
- A Knowledge-Injected Curriculum Pretraining Framework for Question AnsweringXin Lin, Tianhuang Su, Zhenya Huang, Shangzi Xue et al.WWW 2024 · 3 citations
- Masked Language Models Know Which are Popular: A Simple Ranking Strategy for Commonsense Question AnsweringXuan Luo, Chuang Fan, Yice Zhang, Wanguo Jiang et al.EMNLP Findings 2022 · 1 citation
