Reembedding and Reweighting are Needed for Tail Item Sequential Recommendation

Applying large vision models (LVMs) and large language models (LLMs) for item embedding is becoming cutting-edge for sequential recommendation, given their success in broad applications. Despite their advantages over traditional approaches, these models suffer more significant performance degradation on tail items against conventional ID-based solutions, which are largely overlooked by recent research. In this paper, we substantiate the above challenges as (1) all-in ground-truth, i.e., the standard cross-entropy (CE) loss focuses solely on the target items while treating all non-ground-truth equally, causing insufficient optimization for tail items, and (2) knowledge transfer tax, i.e., the knowledge encapsulated in LLMs and LVMs dominates the optimization process due to insufficient training for tail items. We propose Rewarding and reembedding, a simple yet efficient method to address the above challenges. Specifically, we reinitialize tail item embedding via a Gaussian distribution to alleviate knowledge transfer tax; besides, a rewarding function is incorporated in the CE loss, which adaptively adjusts item rewards during training to encourage the model to pay more attention to tail items rather than exclusively optimizing for ground-truth. Overall, our method enables a more nuanced optimization and is mathematically comparable to the direct preference optimization (DPO) in LLMs. Our extensive experiments on three public datasets show our method outperforms fourteen baselines in overall performance and improves the performance on tail items by a large margin. Our code is available at https://github.com/Yuhanleeee/R2Rec.