Efficient privacy loss accounting for subsampling and random allocation
Vitaly Feldman, Moshe Shenfeld
Abstract
We consider the privacy amplification properties of a sampling scheme in which a user’s data is used in steps chosen randomly and uniformly from a sequence (or set) of steps. This sampling scheme has been recently applied in the context of differentially private optimization [Chua et al., 2024a, Choquette-Choo et al., 2025] and communication-efficient high-dimensional private aggregation [Asi et al., 2025], where it was shown to have utility advantages over the standard Poisson sampling. Theoretical analyses of this sampling scheme [Feldman and Shenfeld, 2025, Dong et al., 2025] lead to bounds that are close to those of Poisson sampling, yet still have two significant shortcomings. First, in many practical settings, the resulting privacy parameters are not tight due to the approximation steps in the analysis. Second, the computed parameters are either the hockey stick or Rényi divergence, both of which introduce overheads when used in privacy loss accounting. In this work, we demonstrate that the privacy loss distribution (PLD) of random allocation applied to any differentially private algorithm can be computed efficiently. When applied to the Gaussian mechanism, our results demonstrate that the privacy-utility trade-off for random allocation is at least as good as that of Poisson subsampling. In particular, random allocation is better suited for training via DP-SGD. To support these computations, our work develops new tools for general privacy loss accounting based on a notion of PLD realization. This notion allows us to extend accurate privacy loss accounting to subsampling which previously required manual noise-mechanism-specific analysis.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers1
Ask how each one uses itBuilds on14
- Deep Learning with Differential PrivacyMartín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan et al.CCS 2016 · 7,620 citations
- Numerical Composition of Differential PrivacySivakanth Gopi, Yin Tat Lee, Lukas WutschitzNeurIPS 2021 · 259 citations
- Privacy Amplification via Random Check-InsBorja Balle, Peter Kairouz, Brendan McMahan, Om Dipakbhai Thakkar et al.NeurIPS 2020 · 86 citations
- Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by ShufflingVitaly Feldman, Audra McMillan, Kunal TalwarFOCS 2021 · 76 citations
- Tight on Budget?: Tight Bounds for r-Fold Approximate Differential PrivacySebastian Meiser, Esfandiar MohammadiCCS 2018 · 61 citations
Related papers
- Privacy amplification by random allocationMoshe Shenfeld, Vitaly FeldmanNeurIPS 2025 · 18 citations
- Individual Privacy Accounting with Gaussian Differential PrivacyAntti Koskela, Marlon Tobaben, Antti HonkelaICLR 2023 · 2 citations
- How Private are DP-SGD Implementations?Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar et al.ICML 2024 · 25 citations
- The Skellam Mechanism for Differentially Private Federated LearningNaman Agarwal, Peter Kairouz, Ziyu LiuNeurIPS 2021 · 161 citations
- Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without ReplacementJeremiah Birrell, Reza Ebrahimi, Rouzbeh Behnia, Jason PachecoNeurIPS 2024 · 10 citations
