The formal privacy guarantee provided by Differential Privacy (DP) bounds the leakage of sensitive information from deep learning models. In practice, however, this comes at a severe computation and accuracy cost. The recently established state of the art (SOTA) results in image classification under DP are due to the use of heavy data augmentation and large batch sizes, leading to a drastically increased computation overhead. In this work, we propose to use more efficient models with improved feature quality by introducing steerable equivariant convolutional networks for DP training. We demonstrate that our models are able to outperform the current SOTA performance on CIFAR-10 by up to $9\%$ across different $\varepsilon$-values while reducing the number of model parameters by a factor of $35$ and decreasing the computation time by more than $90 \%$. Our results are a large step towards efficient model architectures that make optimal use of their parameters and bridge the privacy-utility gap between private and non-private deep learning for computer vision.
翻译:“差异隐私”(DP)提供的正式隐私保障使敏感信息从深层学习模式中渗漏出来,但在实践中,这造成了严重的计算和准确性成本;最近建立的最先进的(SOTA)在DP下导致图像分类的原因是使用重数据扩增和大批量尺寸,导致计算间接费用急剧增加;在这项工作中,我们建议采用效率更高的模型,提高特征质量,为DP培训引入可控的等同相动网络;我们证明,我们的模型能够在不同的美元价值中超过目前在CIFAR-10上的SOTA业绩9美元,同时将模型参数数量减少35美元,将计算时间减少90美元以上;我们的成果是朝着高效模型结构迈出的一大步,这些模型能够最佳地利用参数,缩小私人和非私人深层学习用于计算机视觉的私利利用差距。