The area under the ROC curve (AUROC) has been vigorously applied for imbalanced classification and moreover combined with deep learning techniques. However, there is no existing work that provides sound information for peers to choose appropriate deep AUROC maximization techniques. In this work, we fill this gap from three aspects. (i) We benchmark a variety of loss functions with different algorithmic choices for deep AUROC optimization problem. We study the loss functions in two categories: pairwise loss and composite loss, which includes a total of 10 loss functions. Interestingly, we find composite loss, as an innovative loss function class, shows more competitive performance than pairwise loss from both training convergence and testing generalization perspectives. Nevertheless, data with more corrupted labels favors a pairwise symmetric loss. (ii) Moreover, we benchmark and highlight the essential algorithmic choices such as positive sampling rate, regularization, normalization/activation, and optimizers. Key findings include: higher positive sampling rate is likely to be beneficial for deep AUROC maximization; different datasets favors different weights of regularizations; appropriate normalization techniques, such as sigmoid and $\ell_2$ score normalization, could improve model performance. (iii) For optimization aspect, we benchmark SGD-type, Momentum-type, and Adam-type optimizers for both pairwise and composite loss. Our findings show that although Adam-type method is more competitive from training perspective, but it does not outperform others from testing perspective.
翻译:ROC 曲线(AUROC) 下的领域被积极应用于不平衡的分类,并且与深层次学习技术相结合。然而,目前没有一项工作为同行提供可靠的信息,以便选择适当的深AUROC最大化技术。在这项工作中,我们从三个方面填补了这一差距。 (一) 我们用对深AUROC优化问题的不同算法选择来衡量各种损失功能。我们从两类中研究损失功能:双向损失和复合损失,其中包括总共10个损失功能。有趣的是,我们发现,作为创新损失功能类的复合损失显示,从培训趋同和测试通用角度来看,均比双向损失表现出更具有竞争力的绩效。然而,带有更腐败标签的数据有利于双向的对称性类损失。 (二) 此外,我们衡量和突出基本的算法选择,如积极的采样率、正常化、正常化、正常化/活化以及优化。 关键的调查结果包括:较高的正率率可能有利于深层AUROC 最大化;不同的数据集有利于不同的正规化重量;适当的正常化技术,例如Sigmroformal-rual rudeal exal exalal-bilization exaltiewestaltiewslev and wesleval- stration- strationaltiewslevaltiews) (我们的标准、正正正正型、Sirviewxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx