In this paper we consider a problem known as multi-task learning, consisting of fitting a set of classifier or regression functions intended for solving different tasks. In our novel formulation, we couple the parameters of these functions, so that they learn in their task specific domains while staying close to each other. This facilitates cross-fertilization in which data collected across different domains help improving the learning performance at each other task. First, we present a simplified case in which the goal is to estimate the means of two Gaussian variables, for the purpose of gaining some insights on the advantage of the proposed cross-learning strategy. Then we provide a stochastic projected gradient algorithm to perform cross-learning over a generic loss function. If the number of parameters is large, then the projection step becomes computationally expensive. To avoid this situation, we derive a primal-dual algorithm that exploits the structure of the dual problem, achieving a formulation whose complexity only depends on the number of tasks. Preliminary numerical experiments for image classification by neural networks trained on a dataset divided in different domains corroborate that the cross-learned function outperforms both the task-specific and the consensus approaches.
翻译:在本文中,我们考虑一个称为多任务学习的问题,包括安装一套旨在解决不同任务的分类或回归功能。在我们的新配方中,我们将这些功能的参数对齐,以便他们在任务特定领域学习,同时相互接近。这有利于相互交流,在不同领域收集的数据有助于改进彼此任务的学习绩效。首先,我们提出了一个简化案例,目的是估计两个高斯变量的手段,目的是获得对拟议交叉学习战略优势的一些了解。然后,我们提供一个随机预测梯度算法,用于对通用损失函数进行交叉学习。如果参数数量很大,那么预测步骤就变得计算昂贵。为了避免这种情况,我们得出一种原始的双重算法,利用双重问题的结构,实现一种只取决于任务数目的公式。通过在不同领域进行分解的数据集培训的神经网络进行图像分类的初步数字实验,证实了交叉获取的功能超越了任务特定和协商一致方法。