Infinitesimal gradient boosting is defined as the vanishing-learning-rate limit of the popular tree-based gradient boosting algorithm from machine learning (Dombry and Duchamps, 2021). It is characterized as the solution of a nonlinear ordinary differential equation in a infinite-dimensional function space where the infinitesimal boosting operator driving the dynamics depends on the training sample. We consider the asymptotic behavior of the model in the large sample limit and prove its convergence to a deterministic process. This infinite population limit is again characterized by a differential equation that depends on the population distribution. We explore some properties of this population limit: we prove that the dynamics makes the test error decrease and we consider its long time behavior.
翻译:无限梯度推升被定义为机器学习(Dombry和Duchamps, 2021年)中流行的树基梯度推动算法的消失-学习率极限。它被描述为在一个无限功能空间中的非线性普通差异方程式的解决方案,在这个空间中,驱动动态的无限微量助推操作器取决于培训样本。我们考虑模型在大样本限中的无症状行为,并证明它与确定性过程趋同。这个无限的人口限又以取决于人口分布的差别方程式为特征。我们探索了这个人口限的某些特性:我们证明该动态使测试错误减少,我们考虑它的长期行为。