Contrastive learning has become a new paradigm for unsupervised sentence embeddings. Previous studies focus on instance-wise contrastive learning, attempting to construct positive pairs with textual data augmentation. In this paper, we propose a novel Contrastive learning method with Prompt-derived Virtual semantic Prototypes (ConPVP). Specifically, with the help of prompts, we construct virtual semantic prototypes to each instance, and derive negative prototypes by using the negative form of the prompts. Using a prototypical contrastive loss, we enforce the anchor sentence embedding to be close to its corresponding semantic prototypes, and far apart from the negative prototypes as well as the prototypes of other sentences. Extensive experimental results on semantic textual similarity, transfer, and clustering tasks demonstrate the effectiveness of our proposed model compared to strong baselines. Code is available at https://github.com/lemon0830/promptCSE.
翻译:对比性学习已成为未经监督的句子嵌入的新范例。 先前的研究侧重于实例角度的对比性学习, 试图在文本数据增强的同时构建正对。 在本文中, 我们提出了与快速生成的虚拟语义原型( ConPVP ) 的新型对比性学习方法。 具体地说, 在提示的帮助下, 我们在每个实例中构建虚拟语义原型, 并使用负面的提示形式生成负原型。 我们使用原型反比性损失, 强制执行嵌入锚句, 以接近相应的语义原型, 并且远离负原型和其他句子原型。 语义文本相似性、 转移和集群任务方面的广泛实验结果展示了我们提议的模型相对于强势基线的有效性 。 代码可在 https://github. com/lemon0830/ promptCSEEE 上查阅 。