Acceleration and design of homomorphically encrypted convolutional neural networks

Abstract

In recent years, there has been a steady increase in the utilisation of third-party servers equipped with large computational resource for computation outsourcing. This has raised serious concerns regarding data security and privacy. A potential solution is homomorphic encryption, which allows operations to be performed directly on encrypted data, thereby ensuring that the underlying data is never exposed during computation. However, the practicality of homomorphic encryption remains restricted because homomorphic operations are significantly more computationally expensive than their plaintext counterparts. This is especially challenging for machine learning applications, which generally involve computation-intensive workloads.

In this thesis, we address the problem of accelerating the homomorphic computation of convolutional neural networks and explore the design of low-latency neural networks evaluated under homomorphic encryption. We begin by focusing on accelerating the homomorphic computation of individual convolutional layers by leveraging batching, which is a capability of certain homomorphic encryption schemes. Batching allows multiple data values to be packed into the same ciphertext for concurrent computation, reducing the number of homomorphic operations required. Additionally, we propose partitioning homomorphically encrypted convolutional neural networks into sub-networks. We consider the search for an optimal partition that balances the communication and computation costs to yield the lowest overall latency. We also introduce a framework to systematically traverse the design space, exploring different ways to partition a network and pack data into ciphertexts, in search for a design that minimises the overall homomorphic latency. To facilitate quick homomorphic latency estimation in the neural architecture search, we develop a tool that extrapolates the latency of small neural networks to predict the homomorphic latency of a given neural network. Our findings demonstrate that applying neural architecture search to homomorphically encrypted neural networks in an homomorphic encryption-aware manner yields significant performance gains when compared to using plaintext proxies in the performance estimation step.