Imposing explicit constraints is relatively new but increasingly pressing in deep learning, stimulated by, e.g., trustworthy AI that performs robust optimization over complicated perturbation sets and scientific applications that need to respect physical laws and constraints. However, it can be hard to reliably solve constrained deep learning problems without optimization expertise. The existing deep learning frameworks do not admit constraints. General-purpose optimization packages can handle constraints but do not perform auto-differentiation and have trouble dealing with nonsmoothness. In this paper, we introduce a new software package called NCVX, whose initial release contains the solver PyGRANSO, a PyTorch-enabled general-purpose optimization package for constrained machine/deep learning problems, the first of its kind. NCVX inherits auto-differentiation, GPU acceleration, and tensor variables from PyTorch, and is built on freely available and widely used open-source frameworks. NCVX is available at https://ncvx.org, with detailed documentation and numerous examples from machine/deep learning and other fields.
翻译:在深层次的学习中,强加明显限制是相对新的,但越来越紧迫的是,深层次的学习,例如,值得信赖的大赦国际对复杂的扰动装置和科学应用进行强有力的优化,这些装置和科学应用需要尊重物理法则和限制;然而,如果没有最优化的专门知识,很难可靠地解决有限的深层次学习问题;现有的深层次学习框架不承认限制;普通用途优化软件包可以处理各种限制,但不能进行自动区分,处理不光滑的问题。在本文中,我们引入了一个新的软件包,称为NCVX, 最初发行的软件包中包含一个求解器PyGRANSO,一个为受限制的机器/深层学习问题提供PyTorrch的通用优化软件包,这是第一种类型的。 NCVX继承了自动差异、GPU加速和PyTorrch的变数,并且建立在可自由获取和广泛使用的开放源框架之上。NCVX可在https://ncvx.org上查阅,其中提供详细的文件和来自机器/深层学习和其他领域的众多实例。