Think, But Don't Tell: Implicit Reasoning for LLM-based Sequential Recommendation via Multi-Teacher Distillation
Weihai Lu, Xiaoxi Cui, Chenke Yin
摘要
Large Language Models (LLMs) demonstrate significant potential in sequential recommendation, and leveraging their Chain-of-Thought (CoT) reasoning capabilities can further unlock profound user preference understanding. However, deploying explicit CoT reasoning in real-world systems faces prohibitive challenges: (i) the conflict between the large model scale required for high-fidelity reasoning and the resource constraints of online services, and (ii) the excessive latency introduced by auto-regressive rationale generation. To address these issues, we propose I Reasoning via Multi-Teacher Distillation (IRMD), a novel framework that 'compiles' the reasoning abilities of large teacher LLMs into a lightweight student Small Language Model (SLM). IRMD first employs a Multi-Teacher CoT Synthesis with Dual-Constraint Rejection Sampling module to generate a high-quality, diverse set of reasoning paths. Subsequently, our Annealing-Scheduled Reasoning Distillation strategy progressively trains the student to internalize this logic, transitioning from mimicking explicit CoT to performing purely implicit reasoning. Extensive experiments on multiple benchmark datasets demonstrate that IRMD significantly outperforms state-of-the-art baselines in both recommendation accuracy and inference efficiency. Our code is accessible at https://github.com/Cxx-0/IRMD.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- Can Small Language Models be Good Reasoners for Sequential Recommendation?Yuling Wang, Changxin Tian, Binbin Hu, Yanhua Yu 等WWW 2024 · 被引用 69 次
- CODI: Compressing Chain-of-Thought into Continuous Space via Self-DistillationZhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu 等EMNLP 2025
- Mentor-KD: Making Small Language Models Better Multi-step ReasonersHojae Lee, Junho Kim, SangKeun LeeEMNLP 2024
- UniCoTT: A Unified Framework for Structural Chain-of-Thought DistillationXianwei Zhuang, Zhihong Zhu, Zhichang Wang, Xuxin Cheng 等ICLR 2025
- Distilling LLM Agent into Small Models with Retrieval and Code ToolsMinki Kang, Jongwon Jeong, Seanie Lee, Jaewoong Cho 等NeurIPS 2025 · 被引用 51 次
