Cerebros: Evading the RPC Tax in Datacenters
Arash Pourhabibi Zarandi, Mark Sutherland, Alexandros Daglis, Babak Falsafi
Abstract
The emerging paradigm of microservices decomposes online services into fine-grained software modules frequently communicating over the datacenter network, often using Remote Procedure Calls (RPCs). Ongoing advancements in the network stack have exposed the RPC layer itself as a bottleneck, that we show accounts for 40–90% of a microservice’s total execution cycles. We break down the underlying modules that comprise production RPC layers and demonstrate, based on prior evidence, that CPUs can only expect limited improvements for such tasks, mandating a shift to hardware to remove the RPC layer as a limiter of microservice performance. Although recently proposed accelerators can efficiently handle a portion of the RPC layer, their overall benefit is limited by unnecessary CPU involvement, which occurs because the accelerators are architected as co-processors under the CPU’s control. Instead, we show that conclusively removing the RPC layer bottleneck requires all of the RPC layer’s modules to be executed by a NIC-attached hardware accelerator. We introduce Cerebros, a dedicated RPC processor that executes the Apache Thrift RPC layer and acts as an intermediary stage between the NIC and the microservice running on the CPU. Our evaluation using the DeathStarBench microservice suite shows that Cerebros reduces the CPU cycles spent in the RPC layer by 37–64 ×, yielding a 1.8–14 × reduction in total cycles expended per microservice request.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
24 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- RingLeader: Efficiently Offloading Intra-Server Orchestration to NICs cites Efficient Scheduling Policies for Microsecond-Scale Tasks
- Cornflakes: Zero-Copy Serialization for Microsecond-Scale Networking cites Optimus Prime: Accelerating Data Transformation in Servers
- Cornflakes: Zero-Copy Serialization for Microsecond-Scale Networking cites Dagger: efficient and fast RPCs in cloud microservices with near-memory reconfigurable NICs
- Cornflakes: Zero-Copy Serialization for Microsecond-Scale Networking cites Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale
- Rebooting Virtual Memory with Midgard cites Optimus Prime: Accelerating Data Transformation in Servers
- Profiling Hyperscale Big Data Processing cites Optimus Prime: Accelerating Data Transformation in Servers
- Profiling Hyperscale Big Data Processing cites Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale
- The nanoPU: A Nanosecond Network Stack for Datacenters cites Optimus Prime: Accelerating Data Transformation in Servers
- Efficient Scheduling Policies for Microsecond-Scale Tasks cites The nanoPU: A Nanosecond Network Stack for Datacenters
Cited by top-tier papers24
- Efficient Scheduling Policies for Microsecond-Scale TasksSarah McClure, Amy Ousterhout, Scott Shenker, Sylvia RatnasamyNSDI 2022 · 43 citations
- Profiling Hyperscale Big Data ProcessingAbraham Gonzalez, Aasheesh Kolli, Samira Manabi Khan, Sihang Liu et al.ISCA 2023 · 30 citations
- RingLeader: Efficiently Offloading Intra-Server Orchestration to NICsJiaxin Lin, Adney Cardoza, Tarannum Khan, Yeonju Ro et al.NSDI 2023 · 26 citations
- Rebooting Virtual Memory with MidgardSiddharth Gupta, Atri Bhattacharyya, Yunho Oh, Abhishek Bhattacharjee et al.ISCA 2021 · 24 citations
- Cornflakes: Zero-Copy Serialization for Microsecond-Scale NetworkingDeepti Raghavan, Shreya Ravi, Gina Yuan, Pratiksha Thaker et al.SOSP 2023 · 18 citations
Builds on8
- Swift: Delay is Simple and Effective for Congestion Control in the DatacenterGautam Kumar, Nandita Dukkipati, Keon Jang, Hassan M. G. Wassel et al.SIGCOMM 2020 · 333 citations
- Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at HyperscaleAkshitha Sriraman, Abhishek DhanotiaASPLOS 2020 · 78 citations
- The nanoPU: A Nanosecond Network Stack for DatacentersStephen Ibanez, Alex Mallery, Serhat Arslan, Theo Jepsen et al.OSDI 2021 · 74 citations
- Dagger: efficient and fast RPCs in cloud microservices with near-memory reconfigurable NICsNikita Lazarev, Shaojie Xiang, Neil Adit, Zhiru Zhang et al.ASPLOS 2021 · 54 citations
- Optimus Prime: Accelerating Data Transformation in ServersArash Pourhabibi Zarandi, Siddharth Gupta, Hussein Kassir, Mark Sutherland et al.ASPLOS 2020 · 43 citations
Related papers
- RpcNIC: Enabling Efficient Datacenter RPC Offloading on PCIe-attached SmartNICsJie Zhang, Hongjing Huang, Xuzheng Chen, Xiang Li et al.HPCA 2025 · 6 citations
- Remote Procedure Call as a Managed System ServiceJingrong Chen, Yongji Wu, Shihan Lin, Yechen Xu et al.NSDI 2023 · 30 citations
- The NEBULA RPC-Optimized ArchitectureMark Sutherland, Siddharth Gupta, Babak Falsafi, Virendra J. Marathe et al.ISCA 2020 · 29 citations
- ALTOCUMULUS: Scalable Scheduling for Nanosecond-Scale Remote Procedure CallsJiechen Zhao, Iris Uwizeyimana, Karthik Ganesan, Mark C. Jeffrey et al.MICRO 2022 · 11 citations
- AccelFlow: Orchestrating an On-Package Ensemble of Fine-Grained Accelerators for MicroservicesJovan Stojkovic, Abraham Farrell, Zhangxiaowen Gong, Christopher J. Hughes et al.HPCA 2026 · 2 citations
