Orion: A Fully Homomorphic Encryption Framework for Deep Learning

Fully Homomorphic Encryption (FHE) has the potential to substantially improve privacy and security by enabling computation directly on encrypted data. This is especially true with deep learning, as today, many popular user services are powered by neural networks in the cloud. One of the major challenges facing wide-scale deployment of FHE-secured neural inference is effectively mapping these networks to FHE primitives. FHE poses many programming challenges including packing large vectors, automatically managing noise via bootstrapping, and translating arbitrary and general-purpose programs to the limited instruction set provided by FHE. These challenges make building large FHE neural networks intractable using the tools available today. In this paper we address these challenges with Orion, a fully-automated framework for private neural inference in FHE. Orion accepts deep neural networks written in PyTorch and translates them into efficient FHE programs. We achieve this by proposing a novel single-shot multiplexed packing strategy for arbitrary convolutions and through a new, efficient technique to automate bootstrap placement. We evaluate Orion on common benchmarks used by the FHE deep learning community and outperform state-of-the-art by $2.38 \times$ on ResNet-20, the largest network they report. Orion extends naturally to larger networks. We demonstrate this by evaluating ResNet-50 on ImageNet and present the first high-resolution homomorphic object detection experiments using a YOLO-v1 model with 139 million parameters. Finally, we open-source our framework Orion at the following repository: https://github.com/baahl-nyu/orion