Privacy-preserving deep neural network (DNN) inference is a necessity in different regulated industries such as healthcare, finance, and retail. Recently, homomorphic encryption (HE) has been used as a method to enable analytics while addressing privacy concerns. HE enables secure predictions over encrypted data. However, there are several challenges related to the use of HE, including DNN size limitations and the lack of support for some operation types. Most notably, the commonly used ReLU activation is not supported under some HE schemes. We propose a structured methodology to replace ReLU with a quadratic polynomial activation. To address the accuracy degradation issue, we use a pre-trained model that trains another HE-friendly model, using techniques such as "trainable activation" functions and knowledge distillation. We demonstrate our methodology on the AlexNet architecture, using the chest X-Ray and CT datasets for COVID-19 detection. Our experiments show that by using our approach, the gap between the F1 score and accuracy of the models trained with ReLU and the HE-friendly model is narrowed down to within a mere 1.1 - 5.3 percent degradation.
翻译:保护隐私的深神经网络(DNN) 的推断是医疗、金融和零售等不同受监管行业所必须的。 最近,同质加密(HE)被作为一种方法用于在解决隐私问题的同时进行分析。他能够对加密数据进行安全预测。然而,在使用HE方面存在若干挑战,包括DNN尺寸限制和某些操作类型缺乏支持。最明显的是,通常使用的RELU激活在一些HE计划下得不到支持。我们提出了一个结构化的方法,用一种四面形多音激活取代ReLU。为了解决准确性降解问题,我们使用了预先培训的模式来培训另一种HE友好型号模型,使用“可控制激活功能”和知识蒸馏等技术。我们展示了我们在AlexNet结构上的方法,使用胸部X射线和CT数据集进行COVID-19探测。我们的实验表明,通过我们的方法,使用RELU所培训的模型的F1分数和精确度与HE友好型模型之间的差距正在缩小到1.1%至5.3%的降解范围。