In this paper we consider the problem learning of variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization problem, while the latter replaces the lower level optimization problem by an algorithm that solves said problem approximately. Both approaches are used in practice, but, unrolling is much simpler from a computational point of view. To analyze and compare the two approaches, we consider a simple toy model, and compute all risks and the respective estimators explicitly. We show that unrolling can be better than the bilevel optimization approach, but also that the performance of unrolling can depend significantly on further parameters, sometimes in unexpected ways: While the stepsize of the unrolled algorithm matters a lot, the number of unrolled iterations only matters if the number is even or odd, and these two cases are notably different.
翻译:在本文中,我们考虑了在通过风险最小化来监督学习的背景下对变式模型进行学习的问题。我们的目标是通过双级优化和算法解动,更深入地了解两种方法,即通过双级优化和算法解动来学习变式模型。前者认为变式模型是低于风险最小化问题的低级优化问题,而后者则用一种能解决所谓问题的方法来取代低级优化问题。这两种方法都在实践中使用,但从计算的角度分析并比较这两种方法更简单得多。为了分析和比较这两种方法,我们考虑一种简单的玩具模型,并明确计算所有风险和相应的估计数字。我们表明,变式模型比双级优化方法要好,但是还表明,变式的性能在很大程度上取决于进一步的参数,有时是出乎意料的:虽然无序算法的逐步化过程很重要,但只有在数字是偶的或奇异的时,解动式的迭代数数量才很重要。这两种情况明显不同。