Aspect-based Sentiment Analysis (ABSA) aims to determine the sentiment polarity towards an aspect. Because of the expensive and limited labelled data, the pretraining strategy has become the de-facto standard for ABSA. However, there always exists severe domain shift between the pretraining and downstream ABSA datasets, hindering the effective knowledge transfer when directly finetuning and making the downstream task performs sub-optimal. To mitigate such domain shift, we introduce a unified alignment pretraining framework into the vanilla pretrain-finetune pipeline with both instance- and knowledge-level alignments. Specifically, we first devise a novel coarse-to-fine retrieval sampling approach to select target domain-related instances from the large-scale pretraining dataset, thus aligning the instances between pretraining and target domains (\textit{First Stage}). Then, we introduce a knowledge guidance-based strategy to further bridge the domain gap at the knowledge level. In practice, we formulate the model pretrained on the sampled instances into a knowledge guidance model and a learner model, respectively. On the target dataset, we design an on-the-fly teacher-student joint fine-tuning approach to progressively transfer the knowledge from the knowledge guidance model to the learner model (\textit{Second Stage}). Thereby, the learner model can maintain more domain-invariant knowledge when learning new knowledge from the target dataset. In the \textit{Third Stage,} the learner model is finetuned to better adapt its learned knowledge to the target dataset. Extensive experiments and analyses on several ABSA benchmarks demonstrate the effectiveness and universality of our proposed pretraining framework. Notably, our pretraining framework pushes several strong baseline models up to the new state-of-the-art records. We release our code and models.
翻译:以外观为基础的感知分析(ABSA) 旨在确定对某方面的情绪极极性。 由于标签数据昂贵且有限, 预培训战略已成为ABSA的脱法标准。 然而, 预培训和下游ABSA数据集之间始终存在着严格的域变, 直接微调时阻碍有效的知识转移, 并使下游任务执行亚最佳化。 为了减轻这种域变, 我们向香草预流中引入了统一的调整前培训框架, 并配有实例和知识级的调整。 具体地说, 我们首先设计了一个新的全方位至全方位的标准检索取样方法, 从大规模预培训数据集中选择目标领域相关案例, 从而将预培训与目标领域(\ textitit{first Stage}) 数据集相匹配。 然后, 我们引入了基于知识的先导战略, 以进一步弥合知识层面的域差距。 在实践中, 我们将样本中预选的模型发展成一个知识指导模型, 以及一个学习者模型。 在目标数据集上, 我们设计了一个更精细的域级的模型, 我们设计了一个在线的模型, 基础的模型, 将一个知识转换了我们的知识转换到学习模型的模型, 我们的模型, 学习模式的模型的模型的模型的模型 向 学习模式的模型 向 学习 学习 学习了我们学习了我们学习了 学习模式的模型的模型的 学习 学习了 学习 学习 学习 学习 学习 学习 。