Meta Learning,元学习,也叫 Learning to Learn(学会学习)。是继Reinforcement Learning(增强学习)之后又一个重要的研究分支。

知识荟萃

综述

基础算法

元强化学习

应用

视觉应用

其它应用

数据集

视频课程

书籍

代码

领域专家


初步版本,水平有限,有错误或者不完善的地方,欢迎大家提建议和补充,会一直保持更新,本文为专知内容组原创内容,未经允许不得转载,如需转载请发送邮件至fangquanyi@gmail.com 或 联系微信专知小助手(Rancho_Fang)

敬请关注http://www.zhuanzhi.ai 和关注专知公众号,获取第一手AI相关知识

最近更新:2019-12-10

VIP内容

基于meta-learning的方法在有噪声标注的图像分类中取得了显著的效果。这类方法往往需要大量的计算资源,而计算瓶颈在于meta-gradient的计算上。本文提出了一种高效的meta-learning更新方式:Faster Meta Update Strategy (FaMUS),加快了meta-learning的训练速度 (减少2/3的训练时间),并提升了模型的性能。首先,我们发现meta-gradient的计算可以转换成一个逐层计算并累计的形式; 并且,meta-learning的更新只需少量层数在meta-gradient就可以完成。基于此,我们设计了一个layer-wise gradient sampler 加在网络的每一层上。根据sampler的输出,模型可以在训练过程中自适应地判断是否计算并收集该层网络的梯度。越少层的meta-gradient需要计算,网络更新时所需的计算资源越少,从而提升模型的计算效率。

并且,我们发现FaMUS使得meta-learning更加稳定,从而提升了模型的性能。最后,我们在有噪声的分类问题以及长尾分类问题都验证了我们方法的有效性。

https://www.zhuanzhi.ai/paper/fda93b750216436e45e6f660ed76776e

成为VIP会员查看完整内容
0
12

最新论文

In past years model-agnostic meta-learning (MAML) has been one of the most promising approaches in meta-learning. It can be applied to different kinds of problems, e.g., reinforcement learning, but also shows good results on few-shot learning tasks. Besides their tremendous success in these tasks, it has still not been fully revealed yet, why it works so well. Recent work proposes that MAML rather reuses features than rapidly learns. In this paper, we want to inspire a deeper understanding of this question by analyzing MAML's representation. We apply representation similarity analysis (RSA), a well-established method in neuroscience, to the few-shot learning instantiation of MAML. Although some part of our analysis supports their general results that feature reuse is predominant, we also reveal arguments against their conclusion. The similarity-increase of layers closer to the input layers arises from the learning task itself and not from the model. In addition, the representations after inner gradient steps make a broader change to the representation than the changes during meta-training.

0
0
下载
预览
Top