Due to their inference, data representation and reconstruction properties, Variational Autoencoders (VAE) have been successfully used in continual learning classification tasks. However, their ability to generate images with specifications corresponding to the classes and databases learned during Continual Learning (CL) is not well understood and catastrophic forgetting remains a significant challenge. In this paper, we firstly analyze the forgetting behaviour of VAEs by developing a new theoretical framework that formulates CL as a dynamic optimal transport problem. This framework proves approximate bounds to the data likelihood without requiring the task information and explains how the prior knowledge is lost during the training process. We then propose a novel memory buffering approach, namely the Online Cooperative Memorization (OCM) framework, which consists of a Short-Term Memory (STM) that continually stores recent samples to provide future information for the model, and a Long-Term Memory (LTM) aiming to preserve a wide diversity of samples. The proposed OCM transfers certain samples from STM to LTM according to the information diversity selection criterion without requiring any supervised signals. The OCM framework is then combined with a dynamic VAE expansion mixture network for further enhancing its performance.
翻译:由于其推论、数据表述和重建工作的特性,在不断学习的分类任务中成功地使用了变化式自动编码器(VAE),但是,他们制作图像的能力与在不断学习(CL)期间学习的班级和数据库的规格相对应,没有很好地理解,灾难性的遗忘仍然是一个重大挑战。在本文件中,我们首先通过开发一个新的理论框架分析VAE的遗忘行为,将CL发展为动态最佳运输问题。这个框架证明与数据可能性相近,而不需要任务信息,并解释在培训过程中如何丧失先前的知识。然后,我们提出一种新的记忆缓冲方法,即在线合作记忆(OCM)框架,其中包括一个持续储存最新样本以便为模型提供未来信息的短期记忆(STM)和一个长期内存(LTM),目的是保存广泛的样本多样性。拟议的OCM将某些样本从STM转移到LTM,按照信息多样性选择标准而不需要任何监督的信号。随后,OCM框架与动态VAE扩展混合物网络相结合,以进一步提高其性。