Lune

ISCA2026Top-tier venue

CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM

Qingyuan Liu, Liyan Chen, Haocheng Wang, Yanning Yang, Dong Du, Zhigang Mao, Naifeng Jing, Yubin Xia, Haibo Chen

Abstract

Attention-FC Disaggregated (AFD) LLM inference systems offload memory-bound Attention operations to memory-rich accelerators (e.g., CPUs, HBM-PIM) while retaining compute-bound Fully-Connected (FC) operations on GPUs. In this paper, we first design a Disaggregated Roofline Model (DRM) to characterize AFD performance, revealing that system throughput is constrained by the accelerator's limiting factor: either memory bandwidth or capacity. We observe that prior AFD systems often overlook these constraints and fail to balance them, leading to resource underutilization or constrained throughput. Therefore, we propose CHIME, the first AFD system integrating DIMM-PIM, which is a case of the new accelerator that strikes the balance with scalable capacity and bandwidth. To address the synchronization challenges inherent to the distributed cooperating DRAM chips in DIMM-PIM, CHIME employs bubble-free pipelining and hybrid-grained re-layout for efficient attention computation. Furthermore, it maximizes cross-device resource utilization via rankset-granular communication-computation overlapping and alignment-predicting scheduling. Evaluations show CHIME achieves up to 5.15×5.15 \times speedup over state-of-theart HBM-PIM solutions.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve cites FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  • Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve cites Orca: A Distributed Serving System for Transformer-Based Generative Models
  • Efficient Memory Management for Large Language Model Serving with PagedAttention cites FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  • Efficient Memory Management for Large Language Model Serving with PagedAttention cites FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
  • Efficient Memory Management for Large Language Model Serving with PagedAttention cites Orca: A Distributed Serving System for Transformer-Based Generative Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on34

  1. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra et al.NeurIPS 2022 · 5,493 citations
  2. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
  3. Orca: A Distributed Serving System for Transformer-Based Generative ModelsGyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim et al.OSDI 2022 · 690 citations
  4. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUYing Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li et al.ICML 2023 · 683 citations
  5. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-ServeAmey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan et al.OSDI 2024 · 537 citations

Related papers

  1. CHIME: A Cache-Efficient and High-Performance Hybrid Index on Disaggregated MemoryXuchuan Luo, Jiacheng Shen, Pengfei Zuo, Xin Wang et al.SOSP 2024 · 10 citations
  2. DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU MultiplexingLei Gao, Chaoyi Jiang, Hossein Entezari Zarch, Daniel Wong et al.ICML 2026 · 4 citations
  3. BlockPIM: Optimizing Memory Management for PIM-enabled Long-Context LLM InferenceZhichun Li, Jun Zhou, Xueqi Li, Ninghui SunDAC 2025 · 3 citations
  4. POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM InferenceAditya K. Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter et al.ASPLOS 2025 · 24 citations
  5. Pulse: Fine-Grained Hierarchical Hashing Index for Disaggregated MemoryGuangyang Deng, Zixiang Yu, Zhirong Shen, Qiangsheng Su et al.HPCA 2026
Dusk over the sea between two cliffs drawn in fine vertical lines