Generalizability of time series forecasting models depends on the quality of model selection. Temporal cross validation (TCV) is a standard technique to perform model selection in forecasting tasks. TCV sequentially partitions the training time series into train and validation windows, and performs hyperparameter optmization (HPO) of the forecast model to select the model with the best validation performance. Model selection with TCV often leads to poor test performance when the test data distribution differs from that of the validation data. We propose a novel model selection method, H-Pro that exploits the data hierarchy often associated with a time series dataset. Generally, the aggregated data at the higher levels of the hierarchy show better predictability and more consistency compared to the bottom-level data which is more sparse and (sometimes) intermittent. H-Pro performs the HPO of the lowest-level student model based on the test proxy forecasts obtained from a set of teacher models at higher levels in the hierarchy. The consistency of the teachers' proxy forecasts help select better student models at the lowest-level. We perform extensive empirical studies on multiple datasets to validate the efficacy of the proposed method. H-Pro along with off-the-shelf forecasting models outperform existing state-of-the-art forecasting methods including the winning models of the M5 point-forecasting competition.
翻译:时间序列预测模型的通用性取决于模型选择的质量。时间跨度验证(TCV)是一种标准技术,用于在预测任务中进行模型选择。TCV将培训时间序列相继分割成火车和验证窗口,并进行预测模型的超参数优化(HPO)以最佳验证性能选择模型。在测试数据分布不同于验证性数据时,与TCV模式的模型选择往往导致测试性能差。我们提出一种新的模型选择方法,即H-Pro,利用往往与时间序列数据集相关的数据等级。一般而言,较高层级的汇总数据与更稀少和(有时)间歇的底层数据相比,具有更高的可预测性和一致性。H-Pro根据从较高层级一组教师模型获得的测试代理性预测,对最低层次学生模型进行高的测试性预测。教师代理性预测的一致性有助于在最低层次选择更好的学生模型。我们就多种数据集进行广泛的实证研究,以验证拟议方法的功效。H-Pro-Pro-Pro-Pro-Pro-Pro-Pro-pro-stis-stisl-stisl-stisl-stisl-sisl-sl-slviewing-slviewd-sl-sl-slviol-sl-sl-s-sl-sl-sl-s-sl-sl-sl-sl-sl-sl-sl-sl-sl-s-s-s-sl-sl-sl-sl-sl-sl-sl-sl-sl-sl-sl-sl-slvi-sl-sl-sl-sl-s-s-s-s-s-s-s-s-s-s-sl-sl-sl-sl-sl-sl-sl-sl-sl-sl-sl-sl-s-s-sl-sl-sl-sl-sl-sl-sl-sl-s-s-s-s-s-s-s-s-s-sl-sl-sl-sl-sl-sl-s-s-s-sl-sl-s