Diverse Client Selection for Federated Learning via Submodular Maximization

In every communication round of federated learning, each client communicates its model updates back to the server which then aggregates them all. The incurred communication cost and overhead between clients and server, however, can be a major bottleneck particularly when the number of clients is large. We, in this paper, propose to select only a small diverse subset of clients, namely those carrying representative gradient information, and we transmit only these updates to the server. Our aim is for updating via only a subset to approximate updating via aggregating all client information. We achieve this by choosing a subset that maximizes a submodular facility location function defined over gradient space. We introduce "federated averaging with diverse client selection (DivFL)". We provide a thorough analysis of its convergence in the heterogeneous settings and apply it both to synthetic and to real datasets. Empirical results show our approach improves learning efficiency and encourages more uniform (i.e., fair) performance across clients.