Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in particular, identifying the tasks that would benefit from co-training remains a challenging design question. In this paper, we analyze the dynamics of information transfer, or transference, across tasks throughout training. Specifically, we develop a similarity measure that can quantify transference among tasks and use this quantity to both better understand the optimization dynamics of multi-task learning as well as improve overall learning performance. In the latter case, we propose two methods to leverage our transference metric. The first operates at a macro-level by selecting which tasks should train together while the second functions at a micro-level by determining how to combine task gradients at each training step. We find these methods can lead to significant improvement over prior work on three supervised multi-task learning benchmarks and one multi-task reinforcement learning paradigm.
翻译:多任务学习可以利用一项任务所学的信息来帮助培训其他任务。尽管有这种能力,但天真配方往往会降低业绩,特别是确定从共同培训中受益的任务,这仍然是一个具有挑战性的设计问题。在本文件中,我们分析了整个培训任务之间信息转移或转移的动态。具体地说,我们开发了一种类似的计量方法,可以量化任务之间的转移,并利用这一数量来更好地了解多任务学习的优化动态,以及改善总体学习业绩。在后一种情况下,我们提出了两种方法来利用我们的转移指标。第一种是在宏观一级运作,选择哪些任务应该一起培训,而第二种职能则在微观一级运作,确定如何将每个培训步骤的任务梯度结合起来。我们发现,这些方法可以大大改进三个受监督的多任务学习基准和一个多任务强化学习模式的先前工作。