The area under the ROC curve (AUC) is one of the most widely used performance measures for classification models in machine learning. However, it summarizes the true positive rates (TPRs) over all false positive rates (FPRs) in the ROC space, which may include the FPRs with no practical relevance in some applications. The partial AUC, as a generalization of the AUC, summarizes only the TPRs over a specific range of the FPRs and is thus a more suitable performance measure in many real-world situations. Although partial AUC optimization in a range of FPRs had been studied, existing algorithms are not scalable to big data and not applicable to deep learning. To address this challenge, we cast the problem into a non-smooth difference-of-convex (DC) program for any smooth predictive functions (e.g., deep neural networks), which allowed us to develop an efficient approximated gradient descent method based on the Moreau envelope smoothing technique, inspired by recent advances in non-smooth DC optimization. To increase the efficiency of large data processing, we used an efficient stochastic block coordinate update in our algorithm. Our proposed algorithm can also be used to minimize the sum of ranked range loss, which also lacks efficient solvers. We established a complexity of $\tilde O(1/\epsilon^6)$ for finding a nearly $\epsilon$-critical solution. Finally, we numerically demonstrated the effectiveness of our proposed algorithms for both partial AUC maximization and sum of ranked range loss minimization.
翻译:ROC 曲线( AUC) 下的区域是用于机器学习中分类模型的最广泛使用的业绩计量之一。 但是,它总结了ROC 空间中所有虚假正率(FPRs)的真正正率(TPRs), 可能包括FPRs, 在某些应用中没有实际关联。 AUC 部分的缩略图仅概括了FPRs的特定范围,因此是许多真实世界局势中更合适的业绩计量。 虽然已经研究了一系列 FPR 中部分AUC优化, 但现有的算法不能对大数据有效, 也不能适用于深层次学习。 为了应对这一挑战, 我们把问题放到一个非偏差的Convex(DC) 方案中, 任何平稳的预测功能( 例如, 深层的神经网络) 的缩略图中, 它使我们能够根据Moreau $的平整流技术, 开发一个高效的梯度的梯度的梯度下降方法。 最近在非偏差的DC 优化中, 为了提高大数据处理的效率, 我们用一个高效的Squal 的缩缩缩缩缩算, 我们用了一个高效的缩算的缩缩程 。