Heterogeneous Personalized Federated Learning by Local-Global Updates Mixing via Convergence Rate

Federated Learning (FL) allows several clients to construct a global machine-learning model without having to share their data. However, FL often faces the challenge of statistical heterogeneity between the client's data. The heterogeneity of the clients will affect the global model's effectiveness. In this paper, utilizing the Domain Adaptation concept, we define an upper bound for the risk of a global model for every single client and theoretically prove that minimizing heterogeneity between clients minimizes this upper bound. We mainly focus on the heterogeneity in the distribution of data across feature spaces of the clients rather than merely considering imbalanced classes, which have mostly been studied in the literature until now. Therefore, with a primary focus on crosssilo settings, we propose a method called FedGenP, which uses server-trained, client-specific generators for knowledge transfer between clients. Each generator aims to provide samples for the corresponding client to minimize its discrepancy with other clients. Experiments conducted on synthetic and real data, along with a theoretical study, support the effectiveness of our method, demonstrating that the generated data helps in constructing a well-generalized global model. In addition, because of the generators, after fine-tuning the global model with both the local and generated data, the clients need not lose the information from others, resulting in wellgeneralized personalized models as well.