Knowledge transferability, or transfer learning, has been widely adopted to allow a pre-trained model in the source domain to be effectively adapted to downstream tasks in the target domain. It is thus important to explore and understand the factors affecting knowledge transferability. In this paper, as the first work, we analyze and demonstrate the connections between knowledge transferability and another important phenomenon--adversarial transferability, \emph{i.e.}, adversarial examples generated against one model can be transferred to attack other models. Our theoretical studies show that adversarial transferability indicates knowledge transferability and vice versa. Moreover, based on the theoretical insights, we propose two practical adversarial transferability metrics to characterize this process, serving as bidirectional indicators between adversarial and knowledge transferability. We conduct extensive experiments for different scenarios on diverse datasets, showing a positive correlation between adversarial transferability and knowledge transferability. Our findings will shed light on future research about effective knowledge transfer learning and adversarial transferability analyses.
翻译:知识转让或转让学习被广泛采用,以便源领域预先培训的模型能够有效地适应目标领域下游任务,因此,必须探讨和理解影响知识转让的因素,在本文件中,作为首项工作,我们分析和展示知识转让与另一个重要的现象-对抗性转让(meph{i.e.})之间的联系,针对一个模型产生的对抗性实例可以转移,攻击其他模型。我们的理论研究表明,对抗性转让表明知识转让,反之亦然。此外,根据理论见解,我们提出了两种实用的对抗性转让指标,作为对抗性转让与知识转让之间的双向指标,我们对不同数据集的不同情景进行广泛的试验,显示对抗性转让与知识转让之间的积极关系。我们的调查结果将说明关于有效知识转让学习和对抗性转让分析的未来研究。