Recent interest in the external validity of prediction models (i.e., the problem of different train and test distributions, known as dataset shift) has produced many methods for finding predictive distributions that are invariant to dataset shifts and can be used for prediction in new, unseen environments. However, these methods consider different types of shifts and have been developed under disparate frameworks, making it difficult to theoretically analyze how solutions differ with respect to stability and accuracy. Taking a causal graphical view, we use a flexible graphical representation to express various types of dataset shifts. Given a known graph of the data generating process, we show that all invariant distributions correspond to a causal hierarchy of graphical operators which disable the edges in the graph that are responsible for the shifts. The hierarchy provides a common theoretical underpinning for understanding when and how stability to shifts can be achieved, and in what ways stable distributions can differ. We use it to establish conditions for minimax optimal performance across environments, and derive new algorithms that find optimal stable distributions. Using this new perspective, we empirically demonstrate that that there is a tradeoff between minimax and average performance.
翻译:最近对预测模型外部有效性的兴趣(即不同的列车和测试分布(称为数据集转换)问题)已经产生了许多方法来寻找预测分布,这些分布对数据集变化是变化性的,可用于在新的、看不见的环境中进行预测。但是,这些方法考虑到不同类型的变化,是在不同的框架下开发的,因此难以从理论上分析在稳定性和准确性方面的解决办法如何不同。从因果图形角度看,我们使用灵活的图形表达方式来表示各种类型的数据集变化。根据已知的数据生成过程图表,我们表明所有变量分布都对应了图形操作者的因果等级,这些分类使图形操作者对变化负有责任。这些等级提供了共同的理论基础,用于了解何时以及如何实现稳定的转变,以及如何以不同的方式稳定分布。我们用它来为整个环境中的微型成轴最佳性性表现创造条件,并得出新的算法,找到最佳的稳定分布。我们从这一新的角度,从实践中可以证明微轴与平均性能之间的平衡。