TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients
Mengdi Wang, Anna Bodonhelyi, Efe Bozkir, Enkelejda Kasneci
Abstract
Federated learning is a distributed collaborative machine learning paradigm that has gained strong momentum in recent years. In federated learning, a central server periodically coordinates models with clients and aggregates the models trained locally by clients without necessitating access to local data. Despite its potential, the implementation of federated learning continues to encounter several challenges, predominantly the slow convergence that is largely due to data heterogeneity. The slow convergence becomes particularly problematic in cross-device federated learning scenarios where clients may be strongly limited by computing power and storage space, and hence counteracting methods that induce additional computation or memory cost on the client side such as auxiliary objective terms and larger training iterations can be impractical. In this paper, we propose a novel federated aggregation strategy, TurboSVM-FL, that poses no additional computation burden on the client side and can significantly accelerate convergence for federated classification task, especially when clients are "lazy" and train their models solely for few epochs for next global aggregation. TurboSVM-FL extensively utilizes support vector machine to conduct selective aggregation and max-margin spread-out regularization on class embeddings. We evaluate TurboSVM-FL on multiple datasets including FEMNIST, CelebA, and Shakespeare using user-independent validation with non-iid data distribution. Our results show that TurboSVM-FL can significantly outperform existing popular algorithms on convergence rate and reduce communication rounds while delivering better test metrics including accuracy, F1 score, and MCC.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Towards Federated Clustering: A Client-wise Private Graph Aggregation Framework cites Faster Adaptive Federated Learning
- Communication-Efficient Adaptive Federated Learning cites SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
- Communication-Efficient Adaptive Federated Learning cites Adaptive Federated Optimization
- Faster Adaptive Federated Learning cites SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
- Faster Adaptive Federated Learning cites Communication-Efficient Adaptive Federated Learning
Cited by top-tier papers1
Builds on6
- SCAFFOLD: Stochastic Controlled Averaging for Federated LearningSai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi et al.ICML 2020 · 3,875 citations
- Adaptive Federated OptimizationSashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett et al.ICLR 2021 · 1,917 citations
- Federated Learning with Only Positive LabelsFelix X. Yu, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv KumarICML 2020 · 124 citations
- Communication-Efficient Adaptive Federated LearningYujia Wang, Lu Lin, Jinghui ChenICML 2022 · 101 citations
- Faster Adaptive Federated LearningXidong Wu, Feihu Huang, Zhengmian Hu, Heng HuangAAAI 2023 · 99 citations
Related papers
- Federated Learning under Heterogeneous and Correlated Client AvailabilityAngelo Rodio, Francescomaria Faticanti, Othmane Marfoq, Giovanni Neglia et al.INFOCOM 2023 · 27 citations
- Local Learning Matters: Rethinking Data Heterogeneity in Federated LearningMatías Mendieta, Taojiannan Yang, Pu Wang, Minwoo Lee et al.CVPR 2022 · 176 citations
- FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model UpdatesXinyi Lu, Hao Zhang, Chenglin Li, Weijia Lu et al.ICML 2025
- HADFL: Heterogeneity-aware Decentralized Federated Learning FrameworkJing Cao, Zirui Lian, Weihong Liu, Zongwei Zhu et al.DAC 2021 · 28 citations
- Breaking the centralized barrier for cross-device federated learningSai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri et al.NeurIPS 2021 · 113 citations
