Lune

KDD2026Top-tier venue

Experimentation for Different Scheduling Policies on Queues: Mixed Differences-in-Q Estimators Based on Little's Law

Nanshan Jia, Ramesh Johari, Nian Si, Zeyu Zheng

Abstract

In data centers, tasks are dispatched to various servers to evenly distribute the workload. When a data center considers implementing a new scheduling algorithm, it typically conducts an A/B test prior to deployment to assess the real-world impact of this new method. However, a straightforward A/B test might be interfered with so-called "Markovian" interference. We utilized the Differences-in-Q estimator, as developed by Farias et al. [2022], and introduced mixed Differences-in-Q estimators grounded in Little's Law. We show that our A/B testing methods significantly reduce bias and variance when testing various scheduling policies. Extensive simulations were conducted under scenarios like non-stationary arrival rates, heterogeneous service rates, and communication delays. These simulations highlight the robustness and efficacy of our A/B testing approach. Our code is publicly available on GitHub. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Markovian Interference in Experiments cites Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood Approach
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on2

  1. Markovian Interference in ExperimentsVivek F. Farias, Andrew A. Li, Tianyi Peng, Andrew ZhengNeurIPS 2022 · 52 citations
  2. Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood ApproachPeter W. Glynn, Ramesh Johari, Mohammad RasouliNeurIPS 2020 · 45 citations

Related papers

  1. Ah-Q: Quantifying and Handling the Interference within a Datacenter from a System PerspectiveYuhang Liu, Xin Deng, Jiapeng Zhou, Mingyu Chen et al.HPCA 2023 · 15 citations
  2. A More Accurate Algorithm Comparison through A/B Testing using Offline Evaluation MethodsKoki Konishi, Masataka Ushiku, Yuta SaitoKDD 2026 · 1 citation
  3. Near-Optimal Experimental Design Under the Budget Constraint in Online PlatformsYongkang Guo, Yuan Yuan, Jinshan Zhang, Yuqing Kong et al.WWW 2023 · 1 citation
  4. A/B Testing in Network Data with Covariate-Adaptive RandomizationJialu Wang, Ping Li, Feifang HuICML 2023 · 6 citations
  5. Detecting Interference in Online Controlled Experiments with Increasing AllocationKevin Han, Shuangning Li, Jialiang Mao, Han WuKDD 2023 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines