On continual single index learning

In this paper, we generalize the problem of single index model to the context of continual learning in which a learner is challenged with a sequence of tasks one by one and the dataset of each task is revealed in an online fashion. We propose a strategy that is able to learn a common single index for all tasks and a specific link function for each task. The common single index allows to transfer the informaton gained from the previous tasks to a new one. We provide a theoretical analysis of our proposed strategy by proving some regret bounds. Moreover, as a by-product from our work to provide an example of a within-task algorithm, we develop a novel online algorithm for learning single index model in an online setting and provide its regret bound.