A new similarity measure for covariate shift with applications to nonparametric regression
Reese Pathak, Cong Ma, Martin J. Wainwright
Abstract
We study covariate shift in the context of nonparametric regression. We introduce a new measure of distribution mismatch between the source and target distributions that is based on the integrated ratio of probabilities of balls at a given radius. We use the scaling of this measure with respect to the radius to characterize the minimax rate of estimation over a family of Hölder continuous functions under covariate shift. In comparison to the recently proposed notion of transfer exponent, this measure leads to a sharper rate of convergence and is more fine-grained. We accompany our theory with concrete instances of covariate shift that illustrate this sharp difference.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers10
- The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate ShiftJingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu et al.NeurIPS 2022 · 29 citations
- Understanding Forgetting in Continual Learning with Linear RegressionMeng Ding, Kaiyi Ji, Di Wang, Jinhui XuICML 2024 · 23 citations
- Towards a Unified Analysis of Kernel-based Methods Under Covariate ShiftXingdong Feng, Xin He, Caixing Wang, Chao Wang et al.NeurIPS 2023 · 17 citations
- Maximum Likelihood Estimation is All You Need for Well-Specified Covariate ShiftJiawei Ge, Shange Tang, Jianqing Fan, Cong Ma et al.ICLR 2024 · 16 citations
- Minimum-Norm Interpolation Under Covariate ShiftNeil Mallinar, Austin Zane, Spencer Frei, Bin YuICML 2024 · 13 citations
Builds on2
- Near-Optimal Linear Regression under Distribution ShiftQi Lei, Wei Hu, Jason D. LeeICML 2021 · 45 citations
- Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural NetworksSeyed Mohammadreza Mousavi Kalan, Zalan Fabian, Salman Avestimehr, Mahdi SoltanolkotabiNeurIPS 2020 · 37 citations
Related papers
- General Quantification of Covariate and Concept ShiftsHongbo Chen, Li XiaICML 2026
- Robust Correction of Sampling Bias using Cumulative Distribution FunctionsBijan Mazaheri, Siddharth Jain, Jehoshua BruckNeurIPS 2020 · 6 citations
- Overparameterization Improves Robustness to Covariate Shift in High DimensionsNilesh Tripuraneni, Ben Adlam, Jeffrey PenningtonNeurIPS 2021 · 50 citations
- Double-Weighting for Covariate Shift AdaptationJosé Ignacio Segovia-Martín, Santiago Mazuelas, Anqi LiuICML 2023 · 9 citations
- Computational Efficiency under Covariate Shift in Kernel Ridge RegressionAndrea Della Vecchia, Arnaud Mavakala Watusadisi, Ernesto De Vito, Lorenzo RosascoNeurIPS 2025 · 5 citations
