Robust Sequential Experimental Design for A/B Testing
Qianglin Wen, Xiangkun Wu, Chengchun Shi, Ting Li, Niansheng Tang, Yingying Zhang, Hongtu Zhu
Abstract
Experimental design has emerged as a powerful approach for improving the sample efficiency of A/B testing, yet existing designs rely critically on correctly specified models. We study robust sequential experimental design under model misspecification and develop a unified framework that covers both contextual bandit and dynamic settings. Theoretically, we prove that our design bounds the worst-case mean squared error of the estimated treatment effect. Empirically, we demonstrate the effectiveness of the proposed approach using synthetic and real-world datasets from a leading technology company.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on12
- Markovian Interference in ExperimentsVivek F. Farias, Andrew A. Li, Tianyi Peng, Andrew ZhengNeurIPS 2022 · 52 citations
- Adaptive Experimental Design with Temporal Interference: A Maximum Likelihood ApproachPeter W. Glynn, Ramesh Johari, Mohammad RasouliNeurIPS 2020 · 45 citations
- An Instrumental Variable Approach to Confounded Off-Policy EvaluationYang Xu, Jin Zhu, Chengchun Shi, Shikai Luo et al.ICML 2023 · 24 citations
- Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision MakingTing Li, Chengchun Shi, Jianing Wang, Fan Zhou et al.NeurIPS 2023 · 21 citations
- A/B Testing in Network Data with Covariate-Adaptive RandomizationJialu Wang, Ping Li, Feifang HuICML 2023 · 6 citations
Related papers
- Non-stationary Experimental Design under Linear TrendsDavid Simchi-Levi, Chonghuan Wang, Zeyu ZhengNeurIPS 2023 · 6 citations
- Integrating Inference and Experimental Design for Contextual Behavioral Model LearningGongtao Zhou, Haoran YuAAAI 2025
- Post-Contextual-Bandit InferenceAurélien Bibaut, Maria Dimakopoulou, Nathan Kallus, Antoine Chambaz et al.NeurIPS 2021 · 58 citations
- Robust Pricing in Dynamic Mechanism DesignYuan Deng, Sébastien Lahaie, Vahab S. MirrokniICML 2020 · 12 citations
- Beyond Basic A/B Testing: Improving Statistical Efficiency for Business GrowthChangshuai Wei, Phuc Nguyen, Benjamin Zelditch, Joyce ChenKDD 2026 · 1 citation
