多任务学习(MTL)是机器学习的一个子领域,可以同时解决多个学习任务,同时利用各个任务之间的共性和差异。与单独训练模型相比,这可以提高特定任务模型的学习效率和预测准确性。多任务学习是归纳传递的一种方法,它通过将相关任务的训练信号中包含的域信息用作归纳偏差来提高泛化能力。通过使用共享表示形式并行学习任务来实现,每个任务所学的知识可以帮助更好地学习其它任务。

VIP内容

多任务学习(Multi-task learning, MTL)旨在通过对多个相关任务的联合学习来提高任务的泛化能力。作为对比,除了联合训练方案,现代元学习允许在测试阶段进行一些不可见的、标签有限的任务,希望能够快速适应它们。尽管MTL和元学习在问题表述上存在细微的差异,但两种学习范式都认为,现有训练任务之间的共享结构可以导致更好的泛化和适应性。本文通过理论分析和实证调查,进一步了解了这两种学习模式之间的密切联系。理论上,我们首先证明了MTL与一类基于梯度的元学习(GBML)算法具有相同的优化公式。然后我们证明了对于具有足够深度的过参数化神经网络,MTL和GBML学习到的预测函数是接近的。特别是,这一结果表明,这两个模型给出的预测是相似的,在相同的看不见的任务。通过实证,我们证实了我们的理论发现,通过适当的实现,MTL可以在一组少样本分类基准上与先进的GBML算法相媲美。由于现有的GBML算法经常涉及代价高昂的二阶两级优化,我们的一阶MTL方法在大型数据集(如微型imagenet)上快了一个数量级。我们相信,这项工作可以帮助弥合这两种学习模式之间的差距,并提供一个计算效率高的替代GBML,也支持快速任务适应。

https://www.zhuanzhi.ai/paper/5d6fac14a84a1a6163d80eb46284b0af

成为VIP会员查看完整内容
0
6

最新内容

The prominence of figurative language devices, such as sarcasm and irony, poses serious challenges for Arabic Sentiment Analysis (SA). While previous research works tackle SA and sarcasm detection separately, this paper introduces an end-to-end deep Multi-Task Learning (MTL) model, allowing knowledge interaction between the two tasks. Our MTL model's architecture consists of a Bidirectional Encoder Representation from Transformers (BERT) model, a multi-task attention interaction module, and two task classifiers. The overall obtained results show that our proposed model outperforms its single-task counterparts on both SA and sarcasm detection sub-tasks.

0
0
下载
预览

最新论文

The prominence of figurative language devices, such as sarcasm and irony, poses serious challenges for Arabic Sentiment Analysis (SA). While previous research works tackle SA and sarcasm detection separately, this paper introduces an end-to-end deep Multi-Task Learning (MTL) model, allowing knowledge interaction between the two tasks. Our MTL model's architecture consists of a Bidirectional Encoder Representation from Transformers (BERT) model, a multi-task attention interaction module, and two task classifiers. The overall obtained results show that our proposed model outperforms its single-task counterparts on both SA and sarcasm detection sub-tasks.

0
0
下载
预览
父主题
Top