Orion: A Fully Homomorphic Encryption Framework for Deep Learning
Austin Ebel, Karthik Garimella, Brandon Reagen
Abstract
Fully Homomorphic Encryption (FHE) has the potential to substantially improve privacy and security by enabling computation directly on encrypted data. This is especially true with deep learning, as today, many popular user services are powered by neural networks in the cloud. Beyond its well-known high computational costs, one of the major challenges facing wide-scale deployment of FHE-secured neural inference is effectively mapping these networks to FHE primitives. FHE poses many programming challenges including packing large vectors, managing accumulated noise, and translating arbitrary and general-purpose programs to the limited instruction set provided by FHE. These challenges make building large FHE neural networks intractable using the tools available today. In this paper we address these challenges with Orion, a fully-automated framework for private neural inference using FHE. Orion accepts deep neural networks written in PyTorch and translates them into efficient FHE programs. We achieve this by proposing a novel single-shot multiplexed packing strategy for arbitrary convolutions and through a new, efficient technique to automate bootstrap placement and scale management. We evaluate Orion on common benchmarks used by the FHE deep learning community and outperform state-of-the-art by 2.38 × on ResNet-20, the largest network they report. Orion's techniques enable processing much deeper and larger networks. We demonstrate this by evaluating ResNet-50 on ImageNet and present the first high-resolution FHE object detection experiments using a YOLO-v1 model with 139 million parameters. Orion is open-source for all to use at: ://github.com/baahl-nyu/orion https://github.com/baahl-nyu/orion.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
14 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
- BTS: an accelerator for bootstrappable fully homomorphic encryption cites F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption
- BTS: an accelerator for bootstrappable fully homomorphic encryption cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
- Leveraging ASIC AI Chips for Homomorphic Encryption cites F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption
- Leveraging ASIC AI Chips for Homomorphic Encryption cites BTS: an accelerator for bootstrappable fully homomorphic encryption
- Leveraging ASIC AI Chips for Homomorphic Encryption cites CraterLake: a hardware accelerator for efficient unbounded computation on encrypted data
- Leveraging ASIC AI Chips for Homomorphic Encryption cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
- zkPHIRE: A Programmable Accelerator for ZKPs over HIgh-degRee, Expressive Gates cites F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption
- zkPHIRE: A Programmable Accelerator for ZKPs over HIgh-degRee, Expressive Gates cites BTS: an accelerator for bootstrappable fully homomorphic encryption
- zkPHIRE: A Programmable Accelerator for ZKPs over HIgh-degRee, Expressive Gates cites Need for zkSpeed: Accelerating HyperPlonk for Zero-Knowledge Proofs
- zkPHIRE: A Programmable Accelerator for ZKPs over HIgh-degRee, Expressive Gates cites CraterLake: a hardware accelerator for efficient unbounded computation on encrypted data
- Bridging Usability and Performance: A Tensor Compiler for Autovectorizing Homomorphic Encryption cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
- Need for zkSpeed: Accelerating HyperPlonk for Zero-Knowledge Proofs cites F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption
- Need for zkSpeed: Accelerating HyperPlonk for Zero-Knowledge Proofs cites BTS: an accelerator for bootstrappable fully homomorphic encryption
- Need for zkSpeed: Accelerating HyperPlonk for Zero-Knowledge Proofs cites CraterLake: a hardware accelerator for efficient unbounded computation on encrypted data
- CraterLake: a hardware accelerator for efficient unbounded computation on encrypted data cites F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption
- CraterLake: a hardware accelerator for efficient unbounded computation on encrypted data cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
- GAZELLE: A Low Latency Framework for Secure Neural Network Inference cites SecureML: A System for Scalable Privacy-Preserving Machine Learning
Cited by top-tier papers14
- MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer InferenceLinru Zhang, Xiangning Wang, Sim Jun Jie, Zhicong Huang et al.ICLR 2026 · 24 citations
- Need for zkSpeed: Accelerating HyperPlonk for Zero-Knowledge ProofsAlhad Daftardar, Jianqiao Mo, Joey Ah-kiow, Benedikt Bünz et al.ISCA 2025 · 12 citations
- Bridging Usability and Performance: A Tensor Compiler for Autovectorizing Homomorphic EncryptionEdward Chen, Fraser Brown, Wenting ZhengUSENIX Security 2026 · 3 citations
- Leveraging ASIC AI Chips for Homomorphic EncryptionJianming Tong, Tianhao Huang, Jingtian Dang, Leo de Castro et al.HPCA 2026 · 2 citations
- zkPHIRE: A Programmable Accelerator for ZKPs over HIgh-degRee, Expressive GatesAlhad Daftardar, Jianqiao Mo, Joey Ah-kiow, Benedikt Bünz et al.HPCA 2026 · 1 citation
Builds on21
- SecureML: A System for Scalable Privacy-Preserving Machine LearningPayman Mohassel, Yupeng ZhangS&P 2017 · 2,107 citations
- GAZELLE: A Low Latency Framework for Secure Neural Network InferenceChiraag Juvekar, Vinod Vaikuntanathan, Anantha P. ChandrakasanUSENIX Security 2018 · 1,075 citations
- F1: A Fast and Programmable Accelerator for Fully Homomorphic EncryptionNikola Samardzic, Axel Feldmann, Aleksandar Krastev, Srinivas Devadas et al.MICRO 2021 · 294 citations
- CraterLake: a hardware accelerator for efficient unbounded computation on encrypted dataNikola Samardzic, Axel Feldmann, Aleksandar Krastev, Nathan Manohar et al.ISCA 2022 · 205 citations
- BTS: an accelerator for bootstrappable fully homomorphic encryptionSangpyo Kim, Jongmin Kim, Michael Jaemin Kim, Wonkyung Jung et al.ISCA 2022 · 184 citations
Related papers
- Fenc2: Unifying Data Packing for Efficient Private Inference via Convolution and Architecture-Aware Fragment EncodingRan Ran, Zhaoting Gong, Nuo Xu, Yuanchao Xu et al.ISCA 2026
- Orbit: Optimizing Rescale and Bootstrap Placement with Integer Linear Programming Techniques for Secure InferenceZikai Zhou, William Seo, Edward Chen, Alex Ozdemir et al.USENIX Security 2026
- A Tensor Compiler with Automatic Data Packing for Simple and Efficient Fully Homomorphic EncryptionAleksandar Krastev, Nikola Samardzic, Simon Langowski, Srinivas Devadas et al.PLDI 2024 · 23 citations
- FxHENN: FPGA-based acceleration framework for homomorphic encrypted CNN inferenceYilan Zhu, Xinyao Wang, Lei Ju, Shanqing GuoHPCA 2023 · 39 citations
- Hyena: Balancing Packing, Reuse, and Rotations for Encrypted InferenceSarabjeet Singh, Shreyas Singh, Sumanth Gudaparthi, Xiong Fan et al.S&P 2024 · 7 citations
