ScalaTap: Scalable Outbound Rate Limiting in Public Cloud

Cloud providers limit the outbound traffic rate at cloud gateways for billing and performance guarantees. A scalable and accurate tenant-level outbound rate limiting system is critical. The state-of-the-art solution utilizes a centralized controller to orchestrate distributed rate limiting at gateway servers. However, this solution suffers from an inherent tension between accuracy and scalability due to the difficulty in timely adjusting budget allocations between gateway servers. In this paper, we explore a new paradigm for tenant-level outbound rate limiting by decoupling the budget allocator and demand aggregator, which were originally co-located in the centralized controller. We present ScalaTap, a switch-host co-design system that leverages a programmable switch to collect and aggregate local demands, and then distributes global demands to gateway servers for independent budget computation. As a result, the centralized controller is completely removed, and the expensive communication and computational overheads are also eliminated. ScalaTap provides more flexibility in achieving a reasonable trade-off between scalability and accuracy by adjusting the update interval at will. Comprehensive experiments show that ScalaTap can (1) increase rate limiting accuracy by 15.2%-27.3% under the same number of tenants, and (2) support$\mathbf{8}.\mathbf{3}\times$more tenants under the same accuracy guarantee compared with the state-of-the-art solution.