Runtime-Aware Pipeline for Vertical Federated Learning with Bounded Model Staleness

Vertical federated learning (VFL) enables a privacy-preserving collaboration among various parties to train a global model by melding their geo-distributed data features. Communication has been recognized as the primary bottleneck that impairs training efficiency due to frequent cross-party statistics exchange over wide area network. Existing synchronous VFL works often suffer from excessive communication overhead, while asynchronous schemes may introduce significant model staleness, potentially eroding the learning accuracy. In this paper, we propose BS-VFL, an asynchronous VFL with bounded staleness, to pipeline local computation and statistics transmission, substantially reducing the communication overhead while ensuring favorable model performance. Specifically, all data parties will give precedence to local model updates before generating embeddings to curtail model staleness. By analyzing convergence error, we show that BS-VFL can achieve a comparable result to synchronous VFL. Then, we develop a general framework to derive the closed-form wall-clock time of BS-VFL, offering a measure of its runtime efficiency and highlighting a marked communication reduction. Utilizing this convergence and time analysis, we refine learning parameters to minimize the convergence error for optimizing BS-VFL performance without compromising training efficiency. Extensive experiments on real-world datasets validate the superiority of BS-VFL over leading-edge methods, evidencing a reduction in training duration by 48%-90% while preserving model accuracy.