In this paper, we analyze the landscape of the true loss of a ReLU neural network with one hidden layer. We provide a complete classification of the critical points in the case where the target function is affine. In particular, we prove that local minima and saddle points have to be of a special form and show that there are no local maxima. Our approach is of a combinatorial nature and builds on a careful analysis of the different types of hidden neurons that can occur in a ReLU neural network.
翻译:在本文中,我们分析一个隐蔽层的ReLU神经网络真正丧失的景象。 我们提供目标功能为直线的临界点的完整分类。 特别是, 我们证明当地微型和马鞍点必须具有特殊的形式, 并表明当地没有最高标准。 我们的方法是一种组合性的方法,并且建立在对ReLU神经网络中可能发生的不同类型隐藏神经元的仔细分析的基础上。