Dual Efficient Forecasting Framework for Time Series Data

Time series forecasting has been a quintessential problem in data science for decades, with applications ranging from astronomy to zoology. A long time series may not be necessary in practice to achieve only a desired level of prediction accuracy. This work addresses the following fundamental question: How much recent historical data is required to achieve a targeted percentage of statistical prediction efficiency compared to the full time series data? Consequently, the sequential back subsampling (SBS) method, a novel dual efficient forecasting framework, is proposed to estimate the percentage of most recent historical data that achieves computational efficiency (via subsampling) while maintaining a desired level of prediction accuracy (almost as good as compared to full data). Theoretical justification using the asymptotic prediction theory based on traditional AutoRegressive (AR) Models is provided. This framework has been shown to work for recent machine learning forecasting methods even when the models might be misspecified, with empirical illustration using both simulated data and applications to data on financial stock prices and covid-19. The main conclusion is that only a fraction of the most recent historical data provides near-optimal or even better practically relevant predictive accuracy for a broad class of forecasting methods.