Private Federated Learning with Autotuned Compression
Enayat Ullah, Christopher A. Choquette-Choo, Peter Kairouz, Sewoong Oh
Abstract
We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically adjust the compression rate based on the error induced during training, while maintaining provable privacy guarantees through the use of secure aggregation and differential privacy. Our techniques are provably instance-optimal for mean estimation, meaning that they can adapt to the ``hardness of the problem"with minimal interactivity. We demonstrate the effectiveness of our approach on real-world datasets by achieving favorable compression rates without the need for tuning.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext e5270335-7bc4-4166-9362-224b6ceee173Cited by top-tier papers2
- Sketching for Distributed Deep Learning: A Sharper AnalysisMayank Shrivastava, Berivan Isik, Qiaobo Li, Sanmi Koyejo et al.NeurIPS 2024 · 8 citations
- Robust and Actively Secure Serverless Collaborative LearningNicholas Franzese, Adam Dziedzic, Christopher A. Choquette-Choo, Mark R. Thomas et al.NeurIPS 2023 · 7 citations
Builds on20
- Deep Learning with Differential PrivacyMartín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan et al.CCS 2016 · 7,620 citations
- FetchSGD: Communication-Efficient Federated Learning with SketchingDaniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin et al.ICML 2020 · 425 citations
- The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure AggregationPeter Kairouz, Ziyu Liu, Thomas SteinkeICML 2021 · 291 citations
- Practical and Private (Deep) Learning Without Sampling or ShufflingPeter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar et al.ICML 2021 · 239 citations
- Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified ModelsLiam H. Fowl, Jonas Geiping, Wojciech Czaja, Micah Goldblum et al.ICLR 2022 · 181 citations
Related papers
- Privacy-Aware Compression for Federated Learning Through Numerical Mechanism DesignChuan Guo, Kamalika Chaudhuri, Pierre Stock, Michael G. RabbatICML 2023 · 9 citations
- SoteriaFL: A Unified Framework for Private Federated Learning with Communication CompressionZhize Li, Haoyu Zhao, Boyue Li, Yuejie ChiNeurIPS 2022 · 67 citations
- Differentially Private Learning with Adaptive ClippingGalen Andrew, Om Thakkar, Brendan McMahan, Swaroop RamaswamyNeurIPS 2021 · 425 citations
- Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean EstimationWei-Ning Chen, Dan Song, Ayfer Özgür, Peter KairouzNeurIPS 2023 · 42 citations
- DRIVE: One-bit Distributed Mean EstimationShay Vargaftik, Ran Ben-Basat, Amit Portnoy, Gal Mendelson et al.NeurIPS 2021 · 82 citations
