Modern machine learning has achieved impressive prediction performance, but often sacrifices interpretability, a critical consideration in many problems. Here, we propose Fast Interpretable Greedy-Tree Sums (FIGS), an algorithm for fitting concise rule-based models. Specifically, FIGS generalizes the CART algorithm to simultaneously grow a flexible number of trees in a summation. The total number of splits across all the trees can be restricted by a pre-specified threshold, thereby keeping both the size and number of its trees under control. When both are small, the fitted tree-sum can be easily visualized and written out by hand, making it highly interpretable. A partially oracle theoretical result hints at the potential for FIGS to overcome a key weakness of single-tree models by disentangling additive components of generative additive models, thereby reducing redundancy from repeated splits on the same feature. Furthermore, given oracle access to optimal tree structures, we obtain L2 generalization bounds for such generative models in the case of C1 component functions, matching known minimax rates in some cases. Extensive experiments across a wide array of real-world datasets show that FIGS achieves state-of-the-art prediction performance (among all popular rule-based methods) when restricted to just a few splits (e.g. less than 20). We find empirically that FIGS is able to avoid repeated splits, and often provides more concise decision rules than fitted decision trees, without sacrificing predictive performance. All code and models are released in a full-fledged package on Github \url{https://github.com/csinva/imodels}.
翻译:现代机器学习已经取得了令人印象深刻的预测性业绩,但往往牺牲了可解释性,这是许多问题中的一个关键考虑因素。在这里,我们提议快速解释性贪婪-树脂(FIGS),这是设计简洁的基于规则的模型的算法。具体地说,FIGS将CART算法普遍化,以同时增长灵活的树木数量。所有树木的分解总数都可以受到预先指定的阈值的限制,从而可以控制其树的大小和数量。当两者都小时,装配好的树脂和树脂可以很容易地通过手看到和写出来,从而使其高度可解释性化。一个部分的理论结果暗示FIGS(FIGS)有可能克服单一树型模型的关键弱点,方法是将基因化添加模型的添加成分分解,从而减少同一特性上反复分解的树木的冗余。此外,鉴于对最佳树结构的利用程度,我们可以在C1类组合组件功能中为这种精度模型获得L2一般化的界限,将已知的树脂率与某些案例相匹配,使得它很容易被解释。在不那么广泛的两极的S-IFI规则中进行广泛的实验,而不用地进行。