Lune

SIGIR2026Top-tier venue

Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation

Yifan Liu, Yaokun Liu, Zelin Li, Zhenrui Yue, Ruichen Yao, Yang Zhang, Dong Wang

Abstract

Recent advances in generative recommenders adopt a two-stage paradigm: items are first tokenized into semantic IDs using a pretrained tokenizer, and then large language models (LLMs) are trained to generate the next item via sequence-to-sequence modeling. However, these two stages are optimized for different objectives: semantic reconstruction during tokenizer pretraining versus user interaction modeling during recommender training. This objective misalignment leads to two key limitations: (i) suboptimal static tokenization, where fixed token assignments fail to reflect diverse usage contexts; and (ii) discarded pretrained semantics, where pretrained knowledge—typically from language model embeddings—is overwritten during recommender training on user interactions. To address these limitations, we propose to learn DEcomposed COntextual Token Representations (DECOR), a unified framework that preserves pretrained semantics while enhancing the adaptability of token embeddings. DECOR introduces contextualized token composition to refine token embeddings based on user interaction context, and decomposed embedding fusion that integrates pretrained codebook embeddings with newly learned collaborative embeddings. Experiments on three real-world datasets demonstrate that DECOR consistently outperforms state-of-the-art baselines in recommendation performance. Our code is available at https://github.com/yliuaa/DECOR.git.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders cites Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders
  • Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation cites Autoregressive Image Generation using Residual Quantization
  • Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation cites Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders
  • Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation cites Recommender Systems with Generative Retrieval
  • Recommender Systems with Generative Retrieval cites Autoregressive Image Generation using Residual Quantization
  • Recommender Systems with Generative Retrieval cites Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on10

  1. Recommender Systems with Generative RetrievalShashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan et al.NeurIPS 2023 · 474 citations
  2. Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic EncodersYupeng Hou, Jiacheng Li, Xiangjun Fu, Zhankui He et al.ACL 2026 · 346 citations
  3. Learning Vector-Quantized Item Representation for Transferable Sequential RecommendersYupeng Hou, Zhankui He, Julian J. McAuley, Wayne Xin ZhaoWWW 2023 · 256 citations
  4. Autoregressive Image Generation using Residual QuantizationDoyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho et al.CVPR 2022 · 184 citations
  5. Adapting Large Language Models by Integrating Collaborative Semantics for RecommendationBowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen et al.ICDE 2024 · 132 citations

Related papers

  1. Token-level Collaborative Alignment for LLM-based Generative RecommendationFake Lin, Binbin Hu, Zhi Zheng, Xi Zhu et al.WWW 2026 · 1 citation
  2. Universal Item Tokenization for Transferable Generative RecommendationBowen Zheng, Hongyu Lu, Yu Chen, Wayne Xin Zhao et al.SIGIR 2026
  3. From IDs to Semantics: A Generative Framework for Cross-Domain Recommendation with Adaptive Semantic TokenizationPeiyu Hu, Wayne Lu, Jia WangAAAI 2026 · 5 citations
  4. Reasoning over Semantic IDs Enhances Generative RecommendationYingzhi He, Yan Sun, Junfei Tan, Yuxin Chen et al.KDD 2026 · 15 citations
  5. Generative Recommender with End-to-End Learnable Item TokenizationEnze Liu, Bowen Zheng, Cheng Ling, Lantao Hu et al.SIGIR 2025 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines