Uncovering Feature Interdependencies in High-Noise Environments with Stepwise Lookahead Decision Forests

A "stepwise lookahead" variation of the random forest algorithm is presented for its ability to better uncover feature interdependencies inherent in complex systems. Conventionally, random forests are built from "greedy" decision trees which each consider only one split at a time during their construction. In contrast, the decision trees included in this random forest algorithm each simultaneously consider three split nodes in tiers of depth two. It is demonstrated on synthetic data and financial price time series that the lookahead version significantly outperforms the greedy one if certain non-linear relationships between feature-pairs are present. This outperformance is particularly pronounced in regimes of low signal-to-noise ratio. A long-short trading strategy for copper futures is then backtested by training both greedy and non-greedy random forests to predict the signs of daily price returns. The resulting superior performance of the lookahead algorithm is at least partially explained by the presence of "XOR-like" relationships between long-term and short-term technical indicators. More generally, across all examined datasets, when no such relationships between features are present, performance across random forests is similar. Given its enhanced ability to understand the feature-interdependencies present in complex systems, this lookahead variation is a useful extension to the toolkit of data scientists.