Statistical wisdom suggests that very complex models, interpolating training data, will be poor at predicting unseen examples.Yet, this aphorism has been recently challenged by the identification of benign overfitting regimes, specially studied in the case of parametric models: generalization capabilities may be preserved despite model high complexity.While it is widely known that fully-grown decision trees interpolate and, in turn, have bad predictive performances, the same behavior is yet to be analyzed for Random Forests (RF).In this paper, we study the trade-off between interpolation and consistency for several types of RF algorithms. Theoretically, we prove that interpolation regimes and consistency cannot be achieved simultaneously for several non-adaptive RF.Since adaptivity seems to be the cornerstone to bring together interpolation and consistency, we study interpolating Median RF which are proved to be consistent in the interpolating regime. This is the first result conciliating interpolation and consistency for RF, highlighting that the averaging effect introduced by feature randomization is a key mechanism, sufficient to ensure the consistency in the interpolation regime and beyond.Numerical experiments show that Breiman's RF are consistent while exactly interpolating, when no bootstrap step is involved.We theoretically control the size of the interpolation area, which converges fast enough to zero, giving a necessary condition for exact interpolation and consistency to occur in conjunction.
翻译:统计学智慧表明,非常复杂的模型,即培训数据的内插,在预测不可见的例子方面将很难预测。 。 理论上,我们证明,一些非适应性RF无法同时实现内插制度和一致性。 由于适应性似乎是将内插和一致性结合在一起的基石,我们研究的是,虽然众所周知,成熟的决策树的内插制度,反过来,预测性能不佳,但是对随机森林(Random Forest,RF)的同一行为尚待分析。 在本文件中,我们研究若干类型RF算法的内插和一致性之间的权衡。理论上,我们证明,若干非适应性RF模式无法同时实现内插制度和一致性。Sincentity似乎是将内插和一致性结合在一起的基石。 我们研究的是,在内插制度中,经证明是一贯性的,因此,内插法的内插和内插法的一致性效果是关键机制,足以确保内插系统之间的一致性,而内插的内插,而内插系统又不能完全地显示内插。