Lune

CCS2025Top-tier venue

THOR: Secure Transformer Inference with Homomorphic Encryption

Jungho Moon, Dongwoo Yoo, Xiaoqian Jiang, Miran Kim

Abstract

As large language models are increasingly deployed in cloud environments, privacy concerns have become a significant issue. To address this challenge, we present THOR, a non-interactive framework for secure transformer inference using homomorphic encryption. We first propose efficient matrix multiplication algorithms based on diagonal-major encoding and compact ciphertext packing. We extend these basic algorithms to support plaintext-ciphertext matrix multiplication (PC-MM) using parallel submatrix computation and ciphertext-ciphertext multiplication (CC-MM) with a baby-step giant-step strategy. We also design efficient evaluation strategies for non-linear functions such as softmax, LayerNorm, GELU, and Tanh, by integrating advanced approximation techniques with adaptive iterative methods. Our matrix multiplication algorithms outperform state-of-the-art methods, achieving up to 5.3X speedup in PC-MM for ℝ 768 X 768 X ℝ768X128 over BOLT (Pang et al., IEEE S&P 2024) and 9.7X in CC-MM for 12X (ℝ64X128 X ℝ128X128) over Powerformer (Park et al., Preprint). THOR enables secure inference on the BERT-base model with 128 tokens in 10 minutes on a single GPU, while maintaining comparable accuracy on GLUE tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

13 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption cites BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers
  • Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption cites Secure Outsourced Matrix Computation and Application to Neural Networks
  • Iron: Private Inference on Transformers cites Secure Outsourced Matrix Computation and Application to Neural Networks
  • Iron: Private Inference on Transformers cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
  • MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer Inference cites Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption
  • MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer Inference cites BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers
  • Bridging Usability and Performance: A Tensor Compiler for Autovectorizing Homomorphic Encryption cites Iron: Private Inference on Transformers
  • Bridging Usability and Performance: A Tensor Compiler for Autovectorizing Homomorphic Encryption cites BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers
  • Bridging Usability and Performance: A Tensor Compiler for Autovectorizing Homomorphic Encryption cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
  • Hyperion: Private Token Sampling with Homomorphic Encryption cites Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption
  • Hyperion: Private Token Sampling with Homomorphic Encryption cites MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer Inference
  • BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers cites Iron: Private Inference on Transformers
  • BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers cites Secure Outsourced Matrix Computation and Application to Neural Networks
  • BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
  • Secure Outsourced Matrix Computation and Application to Neural Networks cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
  • CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing cites Iron: Private Inference on Transformers
  • CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing cites BOLT: Privacy-Preserving, Accurate and Efficient Inference for Transformers
  • CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing cites GAZELLE: A Low Latency Framework for Secure Neural Network Inference
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers13

  1. MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer InferenceLinru Zhang, Xiangning Wang, Sim Jun Jie, Zhicong Huang et al.ICLR 2026 · 24 citations
  2. Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic EncryptionDongjin Park, Eunsang Lee, Joon-Woo LeeACL 2025 · 13 citations
  3. CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert RoutingYifan Zhou, Tianshi Xu, Jue Hong, Ye Wu et al.NeurIPS 2025 · 4 citations
  4. Bridging Usability and Performance: A Tensor Compiler for Autovectorizing Homomorphic EncryptionEdward Chen, Fraser Brown, Wenting ZhengUSENIX Security 2026 · 3 citations
  5. Hyperion: Private Token Sampling with Homomorphic EncryptionLawrence Lim, Jiaming Liu, Vikas Kalagi, Divyakant Agrawal et al.ACL 2026 · 1 citation

Builds on10

  1. GAZELLE: A Low Latency Framework for Secure Neural Network InferenceChiraag Juvekar, Vinod Vaikuntanathan, Anantha P. ChandrakasanUSENIX Security 2018 · 1,075 citations
  2. Secure Outsourced Matrix Computation and Application to Neural NetworksXiaoqian Jiang, Miran Kim, Kristin E. Lauter, Yongsoo SongCCS 2018 · 359 citations
  3. Iron: Private Inference on TransformersMeng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing et al.NeurIPS 2022 · 209 citations
  4. On the Security of Homomorphic Encryption on Approximate NumbersBaiyu Li, Daniele MicciancioEUROCRYPT 2021 · 165 citations
  5. BOLT: Privacy-Preserving, Accurate and Efficient Inference for TransformersQi Pang, Jinhao Zhu, Helen Möllering, Wenting Zheng et al.S&P 2024 · 149 citations

Related papers

  1. Tricycle: Private Transformer Inference with Tricyclic EncodingsLawrence Lim, Vikas Kalagi, Julia Novick, Jiaming Liu et al.CCS 2026 · 10 citations
  2. Breaking the Layer Barrier: Remodeling Private Transformer Inference with Hybrid CKKS and MPCTianshi Xu, Wen-jie Lu, Jiangrui Yu, Yi Chen et al.USENIX Security 2025
  3. THE-X: Privacy-Preserving Transformer Inference with Homomorphic EncryptionTianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong et al.ACL Findings 2022 · 86 citations
  4. SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPCJinglong Luo, Yehong Zhang, Zhuo Zhang, Jiaqi Zhang et al.ACL Findings 2024 · 23 citations
  5. Mosformer: Maliciously Secure Three-Party Inference Framework for Large TransformersKe Cheng, Yuheng Xia, Anxiao Song, Jiaxuan Fu et al.CCS 2025
Dusk over the sea between two cliffs drawn in fine vertical lines