Recent increases in the computational demands of deep neural networks (DNNs) have sparked interest in efficient deep learning mechanisms, e.g., quantization or pruning. These mechanisms enable the construction of a small, efficient version of commercial-scale models with comparable accuracy, accelerating their deployment to resource-constrained devices. In this paper, we study the security considerations of publishing on-device variants of large-scale models. We first show that an adversary can exploit on-device models to make attacking the large models easier. In evaluations across 19 DNNs, by exploiting the published on-device models as a transfer prior, the adversarial vulnerability of the original commercial-scale models increases by up to 100x. We then show that the vulnerability increases as the similarity between a full-scale and its efficient model increase. Based on the insights, we propose a defense, $similarity$-$unpairing$, that fine-tunes on-device models with the objective of reducing the similarity. We evaluated our defense on all the 19 DNNs and found that it reduces the transferability up to 90% and the number of queries required by a factor of 10-100x. Our results suggest that further research is needed on the security (or even privacy) threats caused by publishing those efficient siblings.
翻译:最近深神经网络(DNNs)的计算需求增加,引起了人们对高效深层学习机制的兴趣,例如量化或裁剪。这些机制使得能够建造一个小的高效商业规模模型,其精确性相当,加速将其部署到资源受限制的装置中。在本文件中,我们研究了在大型模型的安装变体上出版的大型模型的安全考虑。我们首先表明,对手可以利用“点点”模型来更容易攻击大型模型。在对19个“点”模型的评价中,利用已公布的“点”模型作为转移前的转移,原始商业规模模型的对抗性脆弱性增加了多达100倍。我们然后表明,脆弱性随着全面模型与高效模型的相似性而增加。我们基于洞察,提出“以近似值$-$unpairing”的防御,微调“点”模型可以减少相似性。我们评估了所有19个“点”的“点”模型的防御情况,发现它将原始商业规模模型的可转移性降低到高达100倍。我们的安全性研究所需的安全性为10-100所需的安全系数。我们所需要的10个安全等级系数的查询次数。