In the field of pattern classification, the training of deep learning classifiers is mostly end-to-end learning, and the loss function is the constraint on the final output (posterior probability) of the network, so the existence of Softmax is essential. In the case of end-to-end learning, there is usually no effective loss function that completely relies on the features of the middle layer to restrict learning, resulting in the distribution of sample latent features is not optimal, so there is still room for improvement in classification accuracy. Based on the concept of Predefined Evenly-Distributed Class Centroids (PEDCC), this article proposes a Softmax-free loss function based on predefined optimal-distribution of latent features-POD Loss. The loss function only restricts the latent features of the samples, including the norm-adaptive Cosine distance between the latent feature vector of the sample and the center of the predefined evenly-distributed class, and the correlation between the latent features of the samples. Finally, Cosine distance is used for classification. Compared with the commonly used Softmax Loss, some typical Softmax related loss functions and PEDCC-Loss, experiments on several commonly used datasets on several typical deep learning classification networks show that the classification performance of POD Loss is always significant better and easier to converge. Code is available in https://github.com/TianYuZu/POD-Loss.
翻译:在模式分类领域,对深层次学习分类师的培训大多是端到端学习,而损失功能则是对网络最终产出的限制(其他概率),因此 Softmax的存在至关重要。在端到端学习方面,通常没有完全依赖中层特征的有效损失功能,完全依赖中层特征限制学习,导致样本潜在特征分布不尽理想,因此在分类准确性方面仍有改进的余地。根据预先定义的均匀分布类中心(PEDCC)的概念,本文章提议在预先定义的潜伏特性优化分布的基础上,无软麦损失功能,因此,Softmax的存在是必不可少的。在端到端学习中层学习,包括样本潜在特性矢量与前定义平均分布级中心之间的常规适应性距离,以及样本潜在特征之间的关联性关系。最后,Cosine距离用于分类。与常用的 Softmax损失相比,一些典型的软减缩值-PDLOLO-S-SLOD 通常使用的一些标准性能显示显著的实验性能和高级数据。