Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Data

Because of the lack of expertise, to gain benefits from their data, average users have to upload their private data to cloud servers they may not trust. Due to legal or privacy constraints, most users are willing to contribute only their encrypted data, and lack interests or resources to join deep neural network (DNN) training in cloud. To train a DNN on encrypted data in a completely non-interactive way, a recent work proposes a fully homomorphic encryption (FHE)-based technique implementing all activations by Brakerski-Gentry-Vaikuntanathan (BGV)-based lookup tables. However, such inefficient lookup-table-based activations significantly prolong private training latency of DNNs. In this paper, we propose, Glyph, a FHE-based technique to fast and accurately train DNNs on encrypted data by switching between TFHE (Fast Fully Homomorphic Encryption over the Torus) and BGV cryptosystems. Glyph uses logicoperation-friendly TFHE to implement nonlinear activations, while adopts vectorialarithmetic-friendly BGV to perform multiply-accumulations (MACs). Glyph further applies transfer learning on DNN training to improve test accuracy and reduce the number of MACs between ciphertext and ciphertext in convolutional layers. Our experimental results show Glyph obtains state-of-the-art accuracy, and reduces training latency by 69% ∼ 99% over prior FHE-based privacy-preserving techniques on encrypted datasets. Recent works [2, 3, 4] propose cryptographic schemes to enable privacy-preserving training of DNNs. Private federated learning [4] (FL) is created to decentralize DNN training and enable users to train with their own data locally. QUOTIENT [3] takes advantage of multi-party computation (MPC) to interactively train DNNs on both servers and clients. Both FL and MPC require users to stay online and heavily involve in DNN training. However, in some cases, average users may not have strong interest, powerful hardware, or fast network connections for interactive DNN training [5]. To enable DNN training on encrypted data in a completely non-interactive way, a recent study presents the first fully homomorphic encryption (FHE)-based stochastic gradient descent technique [2], FHESGD. 34th Conference on Neural Information Processing Systems (NeurIPS 2020),