Stochastic differential equations of Langevin-diffusion form have received significant recent, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a conceptual model of the stochastic gradient flow in training over-parametrized models. However, the literature typically assumes smoothness of the potential, whose gradient is the drift term. Nevertheless, there are many problems, for which the potential function is not continuously differentiable, and hence the drift is not Lipschitz-continuous everywhere. This is exemplified by robust losses and Rectified Linear Units in regression problems. In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings. In particular, we show strong existence of the solution, as well as asymptotic minimization of the canonical Free Energy Functional.
翻译:最近,由于在巴伊西亚取样算法和优化机器学习中的基本作用,Langevin-divolution形式的蒸馏式差异方程获得了显著的最近进展。在机器学习中,在培训过度平衡模型时,它们作为随机梯度流的概念模型。然而,文献通常假定潜力是平滑的,其梯度是漂流的术语。然而,有许多问题,其潜在功能无法持续地区别,因此漂流并非到处都是Lipschitz。这表现在回归问题中的强力损失和修正线性线性单位。在本文中,我们展示了一些基础结果,说明在适合机器学习环境的假设下,Langevin-stopchacatic Explace Explaces 的流程和无干扰特性。特别是,我们展示了解决方案的强大存在,以及无孔自由能源功能的抑制性最小化。