Visual Prompting (VP) is an emerging and powerful technique that allows sample-efficient adaptation to downstream tasks by engineering a well-trained frozen source model. In this work, we explore the benefits of VP in constructing compelling neural network classifiers with differential privacy (DP). We explore and integrate VP into canonical DP training methods and demonstrate its simplicity and efficiency. In particular, we discover that VP in tandem with PATE, a state-of-the-art DP training method that leverages the knowledge transfer from an ensemble of teachers, achieves the state-of-the-art privacy-utility trade-off with minimum expenditure of privacy budget. Moreover, we conduct additional experiments on cross-domain image classification with a sufficient domain gap to further unveil the advantage of VP in DP. Lastly, we also conduct extensive ablation studies to validate the effectiveness and contribution of VP under DP consideration.
翻译:视觉提示(VP)是一种新兴而强大的技术,它通过构建训练良好的冻结源模型,实现了对下游任务进行高效适应。本文探讨了VP在差分隐私(DP)中构建引人注目的神经网络分类器的优势。我们将VP和DP的经典训练方法结合起来,证明了它的简单和高效。特别地,我们发现,VP与PATE结合使用,后者是一种利用教师集合的知识转移的最先进的DP训练方法,可以以最小的隐私预算实现最先进的隐私-效用平衡。此外,我们进行了跨域图像分类的额外实验,这种分类存在足够的域差距,以进一步揭示VP在DP中的优势。最后,我们还进行了广泛的消融研究,以验证在DP考虑下VP的有效性和贡献。