Fed2: Feature-Aligned Federated Learning

Federated learning learns from scattered data by fusing collaborative models from local nodes. However, conventional coordinatebased model averaging by FedAvg ignored the random information encoded per parameter and may suffer from structural feature misalignment. In this work, we propose 𝐹𝑒𝑑 2 , a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment across the collaborative models. 𝐹𝑒𝑑 2 is composed of two major designs: First, we design a feature-oriented model structure adaptation method to ensure explicit feature allocation in different neural network structures. Applying the structure adaptation to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process, we then propose a feature paired averaging scheme to guarantee aligned feature distribution and maintain no feature fusion conflicts under either IID or non-IID scenarios. Eventually, 𝐹𝑒𝑑 2 could effectively enhance the federated learning convergence performance under extensive homoand heterogeneous settings, providing excellent convergence speed, accuracy, and computation/communication efficiency. CCS CONCEPTS • Computing methodologies → Distributed algorithms.