DeepHider: A Multi-module and Invisibility Watermarking Scheme for Language Model

With the rapid development of natural language processing (NLP) technology, NLP models have shown great economic value in business. However, the owner's models are vulnerable to the threat of pirated redistribution, which breaks the symmetry relationship between model owners and consumers. Therefore, a model protection mechanism is needed to keep the symmetry from being broken. Currently, language model protection schemes based on black-box verification perform poorly in terms of invisibility of trigger samples, which are easily detected by humans or anomaly detectors and thus prevent verification. To solve this problem, this paper proposes a trigger sample of the triggerless mode for ownership verification. In addition, a thief may replace the classification module for a watermarked model to satisfy its specific classification task and remove the watermark present in the model. Therefore, this paper further proposes a new threat of replacing the model classification module and performing global fine-tuning of the model, and successfully verifies the model ownership through a white-box approach. Meanwhile, we use the properties of blockchain such as tamper-proof and traceability to prevent the ownership statement of thieves. Experiments show that the proposed scheme successfully verifies ownership with 100% watermark verification accuracy without affecting the original performance of the model, and has strong robustness and low False trigger rate.