The decision tree is one of the most popular and classical machine learning models from the 1980s. However, in many practical applications, decision trees tend to generate decision paths with excessive depth. Long decision paths often cause overfitting problems, and make models difficult to interpret. With longer decision paths, inference is also more likely to fail when the data contain missing values. In this work, we propose a new tree model called Cascading Decision Trees to alleviate this problem. The key insight of Cascading Decision Trees is to separate the decision path and the explanation path. Our experiments show that on average, Cascading Decision Trees generate 63.38% shorter explanation paths, avoiding overfitting and thus achieve higher test accuracy. We also empirically demonstrate that Cascading Decision Trees have advantages in the robustness against missing values.
翻译:决策树是1980年代最受欢迎和最经典的机器学习模式之一。 但是,在许多实际应用中,决策树往往产生深度过大的决策路径。 长期决策路径往往造成过多的问题,使模型难以解释。 在较长的决策路径中,当数据包含缺失值时,推论也更有可能失败。 在这项工作中,我们提出了一个新的树模型,叫做递增决定树,以缓解这一问题。 递增决定树的关键洞察力是分离决定路径和解释路径。 我们的实验显示,平均而言,递增决定树产生63.38%的较短的解释路径,避免过度调整,从而实现更高的测试准确性。 我们还从经验上证明,递增决定树在抵御缺失值方面具有优势。