Lune

ASPLOS2026Top-tier venue

Bat: Efficient Generative Recommender Serving with Bipartite Attention

Jie Sun, Shaohang Wang, Zimo Zhang, Zhengyu Liu, Yunlong Xu, Peng Sun, Bo Zhao, Bingsheng He, Fei Wu, Zeke Wang

Abstract

Generative Recommenders (GRs) have recently emerged as promising alternatives to traditional Deep Learning Recommendation Models (DLRMs). Despite their potential, GRs remain computationally expensive in inference, exhibiting compute-bound characteristics similar to the prefill stage of Large Language Model (LLM) inference. Prefix caching can reduce redundant computation by reusing previously constructed KV caches. However, the unique properties of GRs, i.e., highly personalized user profiles and real-time item retrieval, make cache reuse across queries challenging, resulting in limited computational savings.

To address these challenges, we present Bat, an efficient serving system for GRs. The key observation is that the semantics between user and item tokens are permutationinvariant. Building on this, we propose Bipartite Attention, a novel attention mechanism that enables adaptive selection of either the user or the item as the prompt prefix without compromising accuracy, thereby unlocking new opportunities

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving cites Efficient Memory Management for Large Language Model Serving with PagedAttention
  • SGLang: Efficient Execution of Structured Language Model Programs cites Efficient Memory Management for Large Language Model Serving with PagedAttention
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on39

  1. SGLang: Efficient Execution of Structured Language Model ProgramsLianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun et al.NeurIPS 2024 · 1,586 citations
  2. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
  3. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank SystemsRuoxi Wang, Rakesh Shivanna, Derek Zhiyuan Cheng, Sagar Jain et al.WWW 2021 · 793 citations
  4. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model ServingYinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu et al.OSDI 2024 · 646 citations
  5. Recommender Systems with Generative RetrievalShashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan et al.NeurIPS 2023 · 474 citations

Related papers

  1. EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register TokensChaoqun Yang, Xinyu Lin, Wenjie Wang, Yongqi Li et al.KDD 2025 · 1 citation
  2. EPIC: Efficient Position-Independent Caching for Serving Large Language ModelsJunhao Hu, Wenrui Huang, Weidong Wang, Haoyi Wang et al.ICML 2025
  3. LLM Query Scheduling with Prefix Reuse and Latency ConstraintsGregory Dexter, Shao Tang, Ata Fatahi Baarzi, Qingquan Song et al.NeurIPS 2025 · 10 citations
  4. Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentionBin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang et al.USENIX ATC 2024 · 273 citations
  5. Bi-Level Optimization for Generative Recommendation: Bridging Tokenization and GenerationYimeng Bai, Chang Liu, Yang Zhang, Dingxian Wang et al.SIGIR 2026
Dusk over the sea between two cliffs drawn in fine vertical lines