Athena: Accelerating Quantized Convolutional Neural Networks under Fully Homomorphic Encryption

Deep learning under FHE is difficult due to two aspects: (1) formidable amount of ciphertext computations like convolutions, so frequent bootstrapping is inevitable which in turn exacerbates the problem; (2) lack of the support to various non-linear functions in terms of the diversity and accuracy.Previous work primarily used the CKKS-based approach, which requires large parameters and places a heavy burden on the hardware.In this paper, we propose Athena, including a novel framework targeting quantized convolutional neural networks under FHE, and a specialized accelerator to release the maximum potential of the framework.Unlike the classic CKKS-based approach, Athena only requires much smaller parameters, i.e., 2 15 degree and approximately 5 MB ciphertext size.Athena uses a uniform representation, functional bootstrapping, to accurately support any type of activation functions, and is not limited to polynomial approximate fitted functions such as ReLU and sigmoid.We highlight the following results: (1) the accuracy varies by +0.01%/-0.24% compared with the plaintext quantized CNN;(2) the inference performance on the Athena accelerator achieves a speedup of 1.5× to 2.3×, an EDAP improvement of 3.8× to 9.9×, compared with state-of-the-art FHE accelerators.