A Kernel-based Consensual Regression Aggregation Method

In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to flexibly combine individual regression estimators $r_1,r_2,\ldots,r_M$ using a weighted average where the weights are defined based on some kernel function. It may be seen as a kernel smoother method implemented on the features of predictions, given by all the individual estimators, instead of the original inputs. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this configuration asymptotically inherits the consistency property of the basic consistent estimators. Moreover, we propose to numerically learn the key parameter of the method using a gradient descent algorithm for a suitable choice of kernel functions instead of using the classical grid search algorithm. The numerical experiments carried out on several simulated and real datasets suggest that the performance of the method is improved with the introduction of kernel functions.