Private Federated Learning with Autotuned Compression
Enayat Ullah, Christopher A. Choquette-Choo, Peter Kairouz, Sewoong Oh
摘要
We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically adjust the compression rate based on the error induced during training, while maintaining provable privacy guarantees through the use of secure aggregation and differential privacy. Our techniques are provably instance-optimal for mean estimation, meaning that they can adapt to the ``hardness of the problem"with minimal interactivity. We demonstrate the effectiveness of our approach on real-world datasets by achieving favorable compression rates without the need for tuning.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- Sketching for Distributed Deep Learning: A Sharper AnalysisMayank Shrivastava, Berivan Isik, Qiaobo Li, Sanmi Koyejo 等NeurIPS 2024 · 被引用 8 次
- Robust and Actively Secure Serverless Collaborative LearningNicholas Franzese, Adam Dziedzic, Christopher A. Choquette-Choo, Mark R. Thomas 等NeurIPS 2023 · 被引用 7 次
它引用的顶会 Paper20
- Deep Learning with Differential PrivacyMartín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan 等CCS 2016 · 被引用 7,620 次
- FetchSGD: Communication-Efficient Federated Learning with SketchingDaniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin 等ICML 2020 · 被引用 425 次
- The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure AggregationPeter Kairouz, Ziyu Liu, Thomas SteinkeICML 2021 · 被引用 291 次
- Practical and Private (Deep) Learning Without Sampling or ShufflingPeter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar 等ICML 2021 · 被引用 239 次
- Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified ModelsLiam H. Fowl, Jonas Geiping, Wojciech Czaja, Micah Goldblum 等ICLR 2022 · 被引用 181 次
相关 Paper
- Privacy-Aware Compression for Federated Learning Through Numerical Mechanism DesignChuan Guo, Kamalika Chaudhuri, Pierre Stock, Michael G. RabbatICML 2023 · 被引用 9 次
- SoteriaFL: A Unified Framework for Private Federated Learning with Communication CompressionZhize Li, Haoyu Zhao, Boyue Li, Yuejie ChiNeurIPS 2022 · 被引用 67 次
- Differentially Private Learning with Adaptive ClippingGalen Andrew, Om Thakkar, Brendan McMahan, Swaroop RamaswamyNeurIPS 2021 · 被引用 425 次
- Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean EstimationWei-Ning Chen, Dan Song, Ayfer Özgür, Peter KairouzNeurIPS 2023 · 被引用 42 次
- DRIVE: One-bit Distributed Mean EstimationShay Vargaftik, Ran Ben-Basat, Amit Portnoy, Gal Mendelson 等NeurIPS 2021 · 被引用 82 次
