Linear Aggregation in Tree-based Estimators

Regression trees and their ensemble methods are popular methods for non-parametric regression: they combine strong predictive performance with interpretable estimators. In order to improve their utility for locally smooth response surfaces, we study regression trees and random forests with linear aggregation functions. We introduce a new algorithm that finds the best axis-aligned split to fit linear aggregation functions on the corresponding nodes, and we offer a quasilinear time implementation. We apply the algorithm to several simulated and real-world data sets. We showcase its favorable performance in an extensive simulation study, and demonstrate its improved interpretability using a large get-out-the-vote randomized controlled trial. We also provide a software package that implements several tree-based estimators with linear aggregation functions and includes tools for inference.