Cross entropy loss has served as the main objective function for classification-based tasks. Widely deployed for learning neural network classifiers, it shows both effectiveness and a probabilistic interpretation. Recently, after the success of self supervised contrastive representation learning methods, supervised contrastive methods have been proposed to learn representations and have shown superior and more robust performance, compared to solely training with cross entropy loss. However, cross entropy loss is still needed to train the final classification layer. In this work, we investigate the possibility of learning both the representation and the classifier using one objective function that combines the robustness of contrastive learning and the probabilistic interpretation of cross entropy loss. First, we revisit a previously proposed contrastive-based objective function that approximates cross entropy loss and present a simple extension to learn the classifier jointly. Second, we propose a new version of the supervised contrastive training that learns jointly the parameters of the classifier and the backbone of the network. We empirically show that our proposed objective functions show a significant improvement over the standard cross entropy loss with more training stability and robustness in various challenging settings.
翻译:交叉引力损失是基于分类的任务的主要客观功能。 广泛部署用于学习神经网络分类器,它显示了有效性和概率解释。 最近,在自我监督的对比式学习方法成功之后,建议采用监督的对比式方法来学习演示,并显示优于和较强的性能,而仅进行交叉引力损失培训就显得更好和较强。 然而,仍然需要交叉引力损失来训练最后分类层。 在这项工作中,我们调查使用一种目标功能来学习代表器和分类器的可能性,该功能结合对比式学习的稳健性和对交叉引力损失的概率解释。 首先,我们重新审视了先前提出的一种基于对比性的目标功能,该功能近似交叉诱导力损失,并为共同学习分类器提供了简单的扩展。 其次,我们提出了监督性对比式培训的新版本,以共同学习分类器的参数和网络的骨干。我们的经验显示,我们拟议的目标功能表明,在各种富有挑战的环境中,在标准交叉引力损失方面有了显著的改进。