Tight and Robust Private Mean Estimation with Few Users
Shyam Narayanan, Vahab S. Mirrokni, Hossein Esfandiari
Abstract
In this work, we study high-dimensional mean estimation under user-level differential privacy, and design an (ε, δ)-differentially private mechanism using as few users as possible. In particular, we provide a nearly optimal trade-off between the number of users and the number of samples per user required for private mean estimation, even when the number of users is as low as O( 1 ε log 1 δ ). Interestingly, this bound on the number of users is independent of the dimension (though the number of samples per user is allowed to depend polynomially on the dimension), unlike the previous work that requires the number of users to depend polynomially on the dimension. This resolves a problem first proposed by Amin et al. [3] . Moreover, our mechanism is robust against corruptions in up to 49% of the users. Finally, our results also apply to optimal algorithms for privately learning discrete distributions with few users, answering a question of Liu et al. [24], and a broader range of problems such as stochastic convex optimization and a variant of stochastic gradient descent via a reduction to differentially private mean estimation.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers15
- DP-PCA: Statistically Optimal and Differentially Private PCAXiyang Liu, Weihao Kong, Prateek Jain, Sewoong OhNeurIPS 2022 · 38 citations
- User-Level Differential Privacy With Few Examples Per UserBadih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi et al.NeurIPS 2023 · 19 citations
- Robustness Implies Privacy in Statistical EstimationSamuel B. Hopkins, Gautam Kamath, Mahbod Majid, Shyam NarayananSTOC 2023 · 16 citations
- Subspace Recovery from Heterogeneous Data with Non-isotropic NoiseJohn C. Duchi, Vitaly Feldman, Lunjia Hu, Kunal TalwarNeurIPS 2022 · 16 citations
- Algorithms for bounding contribution for histogram estimation under user-level privacyYuhan Liu, Ananda Theertha Suresh, Wennan Zhu, Peter Kairouz et al.ICML 2023 · 14 citations
Builds on10
- Deep Learning with Differential PrivacyMartín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan et al.CCS 2016 · 7,620 citations
- Learning with User-Level PrivacyDaniel Levy, Ziteng Sun, Kareem Amin, Satyen Kale et al.NeurIPS 2021 · 113 citations
- Robust and differentially private mean estimationXiyang Liu, Weihao Kong, Sham M. Kakade, Sewoong OhNeurIPS 2021 · 87 citations
- Instance-optimal Mean Estimation Under Differential PrivacyZiyue Huang, Yuting Liang, Ke YiNeurIPS 2021 · 74 citations
- Differentially Private Clustering: Tight Approximation RatiosBadih Ghazi, Ravi Kumar, Pasin ManurangsiNeurIPS 2020 · 68 citations
Related papers
- On User-Level Private Convex OptimizationBadih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi et al.ICML 2023 · 10 citations
- User-level Private Stochastic Convex Optimization with Optimal RatesRaef Bassily, Ziteng SunICML 2023 · 17 citations
- A Central Limit Theorem for Differentially Private Query AnsweringJinshuo Dong, Weijie J. Su, Linjun ZhangNeurIPS 2021 · 21 citations
- Better Locally Private Sparse Estimation Given Multiple Samples Per UserYuheng Ma, Ke Jia, Hanfang YangICML 2024 · 2 citations
- Learning discrete distributions: user vs item-level privacyYuhan Liu, Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar et al.NeurIPS 2020 · 63 citations
