Watermarking has been proposed as a way to protect the Intellectual Property Rights (IPR) of Deep Neural Networks (DNNs) and track their use. Several methods have been proposed that embed the watermark into the trainable parameters of the network (white box watermarking) or into the input-output mappping implemented by the network in correspondence to specific inputs (black box watermarking). In both cases, achieving robustness against fine tuning, model compression and, even more, transfer learning, is one of the most difficult challenges researchers are trying to face with. In this paper, we propose a new white-box, multi-bit watermarking algorithm with strong robustness properties, including retraining for transfer learning. Robustness is achieved thanks to a new information coding strategy according to which the watermark message is spread across a number of fixed weights, whose position depends on a secret key. The weights hosting the watermark are set prior to training, and are left unchanged throughout the entire training procedure. The distribution of the weights carrying out the message is theoretically optimised to make sure that the watermarked weights are indistinguishable from the other weights, while at the same time keeping their amplitude as large as possible to improve robustness against retraining. We carried out several experiments demonstrating the capability of the proposed scheme to provide high payloads with practically no impact on the network accuracy, at the same time retaining excellent robustness against network modifications an re-use, including retraining for transfer learning.
翻译:已经提议将水标记作为保护深神经网络知识产权(IPR)并跟踪其使用情况的一种方法。 已经提议了几种方法,将水标记嵌入网络的可训练参数( 白箱水标记)或网络在特定投入( 黑箱水标记) 中执行的输入输出映射中。 在这两种情况下, 相对于微调、 模型压缩、 甚至转让学习, 实现稳健性是研究人员试图面对的最困难的挑战之一。 在本文中, 我们提议了一个新的白色箱、 多位水标记算法, 具有很强的强健性, 包括转移学习的再培训。 实现水标记是因为新的信息编码战略, 将水标记信息散布于若干固定的重量中, 其位置取决于一个秘密的密钥。 水标记的权重在培训之前就已设定, 在整个培训过程中保持不变。 执行信息重量的分布是理论上的优化, 以确保水标记的重度在网络的精度上保持精确性, 包括高度的精度, 显示其高度的精确性, 以显示其高度的精确性, 同时, 显示其高度的精确性, 从其他的精确性, 显示其精度, 和高度的精确性。