Frequency-Augmented Mixture-of-Heterogeneous-Experts Framework for Sequential Recommendation

Recently, many efforts have been devoted to building effective sequential recommenders.Despite their effectiveness, these methods typically develop a single model to serve all users.However, our empirical studies reveal that different sequential encoders have intrinsic architectural biases and tend to focus on specific behavioral patterns, i.e., particular frequency range of user behavior sequences.For example, the Self-Attention module is essentially a low-pass filter, focusing on low-frequency information while neglecting the high-frequency details.This evidently limits their ability to capture diverse user patterns, leading to suboptimal recommendations.To tackle this problem, we present FamouSRec, a Frequency-Augmented Mixture-of-Heterogeneous-Experts Framework for personalized Recommendations.Our approach builds an MoEbased recommender system, integrating the strengths of various experts to achieve diversified user modeling.For developing the MoE framework, as the key to our approach, we instantiate experts with various model architectures, aiming to leverage their inherent architectural biases and capture diverse behavioral patterns.For selecting appropriate experts to serve individuals, we introduce a frequency-augmented router.It first identifies frequency components in user behavior sequences that are suited for expert encoding, and then conducts customized routing based on the informativeness of these components.Building on this framework, we further propose two novel contrastive tasks to enhance expert