损失函数,在AI中亦称呼距离函数,度量函数。此处的距离代表的是抽象性的,代表真实数据与预测数据之间的误差。损失函数(loss function)是用来估量你模型的预测值f(x)与真实值Y的不一致程度,它是一个非负实值函数,通常使用L(Y, f(x))来表示,损失函数越小,模型的鲁棒性就越好。损失函数是经验风险函数的核心部分,也是结构风险函数重要组成部分。

VIP内容

题目: Supervised Contrastive Learning

简介: 交叉熵是在图像分类模型的有监督训练中使用最广泛的损失函数。在本文中,我们提出了一种新颖的训练方法,该方法在跨不同体系结构和数据扩充的监督学习任务上始终优于交叉熵。我们修改了批处理的对比损失,最近已证明该方法对于在自我监督的情况下学习强大的表示非常有效。因此,我们能够比交叉熵更有效地利用标签信息。属于同一类别的点的群集在嵌入空间中聚在一起,同时将不同类别的样本群集推开。除此之外,我们还利用了关键成分,例如大批处理量和标准化的嵌入,这些成分已显示出对自我监督学习的好处。在ResNet-50和ResNet-200上,我们的交叉熵均超过1%,在使用AutoAugment数据增强的方法中,新的技术水平达到了78.8%。损失还显示出明显的好处,即可以在标准基准和准确性方面提高对自然基准的自然破坏的鲁棒性。与交叉熵相比,我们监督的对比损失对于诸如优化器或数据增强之类的超参数设置更稳定。

成为VIP会员查看完整内容
0
11

最新内容

Linear mixed models (LMMs) are instrumental for regression analysis with structured dependence, such as grouped, clustered, or multilevel data. However, selection among the covariates--while accounting for this structured dependence--remains a challenge. We introduce a Bayesian decision analysis for subset selection with LMMs. Using a Mahalanobis loss function that incorporates the structured dependence, we derive optimal linear actions for any subset of covariates and under any Bayesian LMM. Crucially, these actions inherit shrinkage or regularization and uncertainty quantification from the underlying Bayesian LMM. Rather than selecting a single "best" subset, which is often unstable and limited in its information content, we collect the acceptable family of subsets that nearly match the predictive ability of the "best" subset. The acceptable family is summarized by its smallest member and key variable importance metrics. Customized subset search and out-of-sample approximation algorithms are provided for more scalable computing. These tools are applied to simulated data and a longitudinal physical activity dataset, and in both cases demonstrate excellent prediction, estimation, and selection ability.

0
0
下载
预览

最新论文

Linear mixed models (LMMs) are instrumental for regression analysis with structured dependence, such as grouped, clustered, or multilevel data. However, selection among the covariates--while accounting for this structured dependence--remains a challenge. We introduce a Bayesian decision analysis for subset selection with LMMs. Using a Mahalanobis loss function that incorporates the structured dependence, we derive optimal linear actions for any subset of covariates and under any Bayesian LMM. Crucially, these actions inherit shrinkage or regularization and uncertainty quantification from the underlying Bayesian LMM. Rather than selecting a single "best" subset, which is often unstable and limited in its information content, we collect the acceptable family of subsets that nearly match the predictive ability of the "best" subset. The acceptable family is summarized by its smallest member and key variable importance metrics. Customized subset search and out-of-sample approximation algorithms are provided for more scalable computing. These tools are applied to simulated data and a longitudinal physical activity dataset, and in both cases demonstrate excellent prediction, estimation, and selection ability.

0
0
下载
预览
Top