Ensemble Projection Pursuit for General Nonparametric Regression

The projection pursuit regression (PPR) has played an important role in the development of statistics and machine learning. According to the two cultures of Breiman (2001), PPR is an algorithmic model that can be used to approximate any general regression. Although PPR can achieve the almost optimal consistency rate asymptotically as shown in this paper, its effectiveness in prediction is rarely seen in practice. To improve the prediction, we propose an ensemble procedure, hereafter referred to as ePPR, by adopting the "feature bagging" of the Random Forest (RF). In comparison, ePPR has several advantages over RF, and its theoretical consistency can be proved under more general settings than RF. Extensive comparisons based on real data sets show that ePPR is significantly more efficient in regression and classification than RF and other competitors.