What is learning? 20$^{st}$ century formalizations of learning theory -- which precipitated revolutions in artificial intelligence -- focus primarily on $\mathit{in-distribution}$ learning, that is, learning under the assumption that the training data are sampled from the same distribution as the evaluation distribution. This assumption renders these theories inadequate for characterizing 21$^{st}$ century real world data problems, which are typically characterized by evaluation distributions that differ from the training data distributions (referred to as out-of-distribution learning). We therefore make a small change to existing formal definitions of learnability by relaxing that assumption. We then introduce $\mathbf{learning\ efficiency}$ (LE) to quantify the amount a learner is able to leverage data for a given problem, regardless of whether it is an in- or out-of-distribution problem. We then define and prove the relationship between generalized notions of learnability, and show how this framework is sufficiently general to characterize transfer, multitask, meta, continual, and lifelong learning. We hope this unification helps bridge the gap between empirical practice and theoretical guidance in real world problems. Finally, because biological learning continues to outperform machine learning algorithms on certain OOD challenges, we discuss the limitations of this framework vis-\'a-vis its ability to formalize biological learning, suggesting multiple avenues for future research.
翻译:什么是学习? 20美元? 20美元? 20美元? 一个世纪的学习理论正规化,它催生了人工智能的革命 -- -- 主要侧重于 $\ mathit{in-ission} learning $,也就是说,根据培训数据样本与评价分布相同这一假设进行学习。这一假设使得这些理论不足以说明21美元 $st} 美元 真正的世界数据问题,其典型特征是评价分布不同于培训数据分布(称为分配外学习)。因此,我们对现有的正式学习定义稍作改变,通过放松这一假设,我们先采用$\mathb{learn\ prolegation\ phiative $(LE) 来量化学习者能够将数据用于某个特定问题的数量,而不管它是否属于分配问题。我们然后界定和证明通用的学习概念之间的关系,并表明这个框架如何足够笼统地描述转让、多重任务、元、持续和终身学习。我们希望这种统一有助于弥合实际世界经验实践与理论指导之间的差距。然后我们引入$_效率。最后,我们学习生物学的系统, 学习生物学成一个未来的系统。