多任务学习(MTL)是机器学习的一个子领域,可以同时解决多个学习任务,同时利用各个任务之间的共性和差异。与单独训练模型相比,这可以提高特定任务模型的学习效率和预测准确性。多任务学习是归纳传递的一种方法,它通过将相关任务的训练信号中包含的域信息用作归纳偏差来提高泛化能力。通过使用共享表示形式并行学习任务来实现,每个任务所学的知识可以帮助更好地学习其它任务。

VIP内容

图神经网络(GNNs)被广泛用于学习一种强大的图结构数据表示。最近的研究表明,将知识从自监督任务迁移到下游任务可以进一步改善图的表示。然而,自监督任务与下游任务在优化目标和训练数据上存在内在的差距。传统的预训练方法可能对知识迁移不够有效,因为它们不能适应下游任务。为了解决这一问题,我们提出了一种新的迁移学习范式,该范式可以有效地将自监督任务作为辅助任务来帮助目标任务。在微调阶段,我们的方法将不同的辅助任务与目标任务进行自适应的选择和组合。我们设计了一个自适应辅助损失加权模型,通过量化辅助任务与目标任务之间的一致性来学习辅助任务的权重。此外,我们通过元学习来学习权重模型。我们的方法可以运用于各种迁移学习方法,它不仅在多任务学习中有很好的表现,而且在预训练和微调中也有很好的表现。在多个下游任务上的综合实验表明,所提出的方法能够有效地将辅助任务与目标任务相结合,与现有的方法相比,显著提高了性能。

https://www.zhuanzhi.ai/paper/852db932624d6feeb7bbd32e67772b27

成为VIP会员查看完整内容
0
22

最新论文

The recognition of hate speech and offensive language (HOF) is commonly formulated as a classification task to decide if a text contains HOF. We investigate whether HOF detection can profit by taking into account the relationships between HOF and similar concepts: (a) HOF is related to sentiment analysis because hate speech is typically a negative statement and expresses a negative opinion; (b) it is related to emotion analysis, as expressed hate points to the author experiencing (or pretending to experience) anger while the addressees experience (or are intended to experience) fear. (c) Finally, one constituting element of HOF is the mention of a targeted person or group. On this basis, we hypothesize that HOF detection shows improvements when being modeled jointly with these concepts, in a multi-task learning setup. We base our experiments on existing data sets for each of these concepts (sentiment, emotion, target of HOF) and evaluate our models as a participant (as team IMS-SINAI) in the HASOC FIRE 2021 English Subtask 1A. Based on model-selection experiments in which we consider multiple available resources and submissions to the shared task, we find that the combination of the CrowdFlower emotion corpus, the SemEval 2016 Sentiment Corpus, and the OffensEval 2019 target detection data leads to an F1 =.79 in a multi-head multi-task learning model based on BERT, in comparison to .7895 of plain BERT. On the HASOC 2019 test data, this result is more substantial with an increase by 2pp in F1 and a considerable increase in recall. Across both data sets (2019, 2021), the recall is particularly increased for the class of HOF (6pp for the 2019 data and 3pp for the 2021 data), showing that MTL with emotion, sentiment, and target identification is an appropriate approach for early warning systems that might be deployed in social media platforms.

0
0
下载
预览
父主题
Top