In domains where sample sizes are limited, efficient learning algorithms are critical. Learning using privileged information (LuPI) offers increased sample efficiency by allowing prediction models access to types of information at training time which is unavailable when the models are used. In recent work, it was shown that for prediction in linear-Gaussian dynamical systems, a LuPI learner with access to intermediate time series data is never worse and often better in expectation than any unbiased classical learner. We provide new insights into this analysis and generalize it to nonlinear prediction tasks in latent dynamical systems, extending theoretical guarantees to the case where the map connecting latent variables and observations is known up to a linear transform. In addition, we propose algorithms based on random features and representation learning for the case when this map is unknown. A suite of empirical results confirm theoretical findings and show the potential of using privileged time-series information in nonlinear prediction.
翻译:在抽样规模有限的领域,使用特许信息(LuPI)的学习提高了抽样效率,允许预测模型在使用模型时无法获得的培训时间访问各类信息。在最近的工作中,发现在线性-Gausian动态系统中的预测中,能够获取中间时间序列数据的LuPI学习者比任何不偏倚的古典学习者更差,也往往更期望获得中间时间序列数据的LuPI学习者。我们对这一分析提供新的洞察,将其概括到潜伏动态系统中的非线性预测任务,将理论保障扩大到将潜在变量和观测结果联系起来的地图变为线性变的情况。此外,我们建议在地图未知的情况下,根据随机特征进行算法和代表学习。一系列经验结果证实了理论结论,并展示了在非线性预测中使用特许时间序列信息的可能性。