EMNLP Findings2025
Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation
Chengbing Wang, Yang Zhang, Zhicheng Wang, Tianhao Shi, Keqin Bao, Fuli Feng, Tat-Seng Chua
Abstract
Fine-tuning large language models (LLMs) for recommendation in a generative manner has delivered promising results, but encounters significant inference overhead due to autoregressive decoding in the language space. This work explores bypassing language-space decoding by directly matching candidate items with the LLM's internal thought representations in the latent space, eliminating the time-consuming autoregressive process to reduce computational costs. Towards this, we introduce Light Latentspace Decoding (L2D), an effective and efficient latent-space decoding method. L2D represents user-preferred items by using the hidden states of test sequences reflecting the LLM's internal thought, and obtains candidate item representations from the hidden states of training sequences labeled with the corresponding candidate items. It then matches the two types of representations to decode items, achieving latent-space decoding. In this way, it enables efficient decoding without altering the LLM's generative tuning paradigm, thereby preserving performance. Extensive empirical results demonstrate that L2D is more than 10x faster than language-space decoding while maintaining or enhancing performance.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on11
- Fast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasICML 2023 · 1,472 citations
- Collaborative Large Language Model for Recommender SystemsYaochen Zhu, Liang Wu, Qi Guo, Liangjie Hong et al.WWW 2024 · 150 citations
- Adapting Large Language Models by Integrating Collaborative Semantics for RecommendationBowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen et al.ICDE 2024 · 132 citations
- Reinforced Latent Reasoning for LLM-based RecommendationYang Zhang, Wenxin Xu, Xiaoyan Zhao, Wenjie Wang et al.ICLR 2026 · 67 citations
- PMG : Personalized Multimodal Generation with Large Language ModelsXiaoteng Shen, Rui Zhang, Xiaoyan Zhao, Jieming Zhu et al.WWW 2024 · 40 citations
Related papers
- Language Ranker: A Lightweight Ranking framework for LLM DecodingChenheng Zhang, Tianqi Du, Jizhe Zhang, Mingqing Xiao et al.NeurIPS 2025 · 3 citations
- Efficient Inference for Large Language Model-based Generative RecommendationXinyu Lin, Chaoqun Yang, Wenjie Wang, Yongqi Li et al.ICLR 2025 · 1 citation
- Reinforcement Speculative Decoding for Fast RankingYingpeng Du, Tianjun Wei, Zhu Sun, Jie Zhang et al.KDD 2026 · 4 citations
- EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register TokensChaoqun Yang, Xinyu Lin, Wenjie Wang, Yongqi Li et al.KDD 2025 · 1 citation
- Rethinking LLM Reasoning: From Explicit Trajectories to Latent RepresentationsCong Jiang, Xiaofeng Zhang, Fangzhi Zhu, XiaoWei Chen et al.ICLR 2026
