FedBridge: Accelerating Edge-Assisted Federated Learning for Model-Heterogeneous Clients

In an edge-assisted federated learning (FL) system, edge servers aggregate client models into intermediate models for the cloud server to produce the global model communication-efficiently. However, existing edge-assisted systems fail to accommodate model-heterogeneous clients, i.e., clients running models with different architectures. To tackle this problem, this paper proposes FedBridge, a novel edge-assisted system that enables FL across model-heterogeneous clients through a two-tier knowledge-sharing mechanism. It deploys an expandable fusion model on each edge server in the system to fuse the knowledge from heterogeneous client models through knowledge distillation. In the meantime, it employs a contrastive loss to mitigate data heterogeneity in client data by aligning the logits of the fusion model close to those of the global model. On the cloud server, it employs a block-based aggregation method to merge fusion models transmitted from the edge servers. We conduct extensive experiments with three models on two widely-used public datasets to evaluate the performance of FedBridge. The results demonstrate that, compared to state-of-the-art systems, FedBridge accelerates model convergence by up to 6.3x and improves model accuracy by 6.2%-17.7%, with 89.51%-96.13% reduction in communication overhead and 48.12%-61.77% in memory overhead.