Meta-learning synthesizes and leverages the knowledge from a given set of tasks to rapidly learn new tasks using very little data. Meta-learning of linear regression tasks, where the regressors lie in a low-dimensional subspace, is an extensively-studied fundamental problem in this domain. However, existing results either guarantee highly suboptimal estimation errors, or require $\Omega(d)$ samples per task (where $d$ is the data dimensionality) thus providing little gain over separately learning each task. In this work, we study a simple alternating minimization method (MLLAM), which alternately learns the low-dimensional subspace and the regressors. We show that, for a constant subspace dimension MLLAM obtains nearly-optimal estimation error, despite requiring only $\Omega(\log d)$ samples per task. However, the number of samples required per task grows logarithmically with the number of tasks. To remedy this in the low-noise regime, we propose a novel task subset selection scheme that ensures the same strong statistical guarantee as MLLAM, even with bounded number of samples per task for arbitrarily large number of tasks.
翻译:元学习合成并利用特定任务组的知识,利用极小的数据快速学习新任务。线性回归任务(回归器位于一个低维次空间)的元学习是该领域一个广泛研究的根本问题。但是,现有结果要么保证了极不完美的估计错误,要么要求每个任务(即美元为数据维度)需要$\Omega(d)的样本,从而在分别学习每项任务方面几乎没有什么收益。在这项工作中,我们研究一种简单的交替最小化方法(MLLAM),以学习低维次空间和回归器。我们表明,对于一个恒定的子空间层面,MLLAM获得几乎最佳的估计错误,尽管每个任务只需要$\Omega(\log d)的样本。然而,每个任务所需的样本数量随着任务数量的增加而成逻辑。为了在低噪音制度中纠正这一点,我们提出了一个新的任务分类选择方案,以确保与MLLLAM一样强大的统计保证,即使每个任务中要求的样本数量有限。