Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization.
翻译:空间时空(ST)图形模型模型,如交通速度预测和出租车需求预测,是深层学习领域的一项重要任务。但是,对于图表中的节点,它们的ST型模式在建模困难方面差异很大,具有ST数据的多样性。我们认为,以有意义的顺序,从容易到复杂,将节点公之于模型,可以比传统的培训程序带来绩效改进。这一理念在课程学习中具有根本意义,它表明在培训模型的早期阶段,对噪音和困难样本可能很敏感。在本文中,我们提出了ST-Curculum 辍学,这是空间时空图模型模型的新颖和易于执行的战略。具体地说,我们评估了高级地物空间中每个节点的学习困难,并将这些困难的节点扔出去,以确保模型只需在开始处理基本的ST关系,然后逐渐转向硬程序。我们的战略可以适用于任何有理论意义的深层次学习结构,而无需额外的训练参数,并且对广泛的数据集进行广泛的实验,通过控制ST-时空关系的难度程度,从而能够更好地测量模型,从而更好地测量总的进展。