SRATTA: Sample Re-ATTribution Attack of Secure Aggregation in Federated Learning

We consider a cross-silo federated learning (FL) setting where a machine learning model with a fully connected first layer is trained between different clients and a central server using FedAvg, and where the aggregation step can be performed with secure aggregation (SA). We present SRATTA an attack relying only on aggregated models which, under realistic assumptions, (i) recovers data samples from the different clients, and (ii) groups data samples coming from the same client together. While sample recovery has already been explored in an FL setting, the ability to group samples per client, despite the use of SA, is novel. This poses a significant unforeseen security threat to FL and effectively breaks SA. We show that SRATTA is both theoretically grounded and can be used in practice on realistic models and datasets. We also propose counter-measures, and claim that clients should play an active role to guarantee their privacy during training. Recently, the efficiency of SA to prevent reconstruction attacks has been questioned, as gradient attacks Zhu et al. [2019] can recover samples from large batches of raw gradients Yin et al. [2021]. In the FL setting, recent attacks Geiping et al. ˚Alphabetical order