QA models with lifelong learning (LL) abilities are important for practical QA applications, and architecture-based LL methods are reported to be an effective implementation for these models. However, it is non-trivial to extend previous approaches to QA tasks since they either require access to task identities in the testing phase or do not explicitly model samples from unseen tasks. In this paper, we propose Diana: a dynamic architecture-based lifelong QA model that tries to learn a sequence of QA tasks with a prompt enhanced language model. Four types of hierarchically organized prompts are used in Diana to capture QA knowledge from different granularities. Specifically, we dedicate task-level prompts to capture task-specific knowledge to retain high LL performances and maintain instance-level prompts to learn knowledge shared across different input samples to improve the model's generalization performance. Moreover, we dedicate separate prompts to explicitly model unseen tasks and introduce a set of prompt key vectors to facilitate knowledge sharing between tasks. Extensive experiments demonstrate that Diana outperforms state-of-the-art lifelong QA models, especially in handling unseen tasks.
翻译:具有终身学习能力的质量保证模型对于实用的质量保证应用十分重要,据报告,基于架构的有限li方法是这些模型的有效实施。然而,推广以前对质量保证任务的做法并非三管齐下,因为它们要么要求在测试阶段获得任务身份,要么没有明确模拟无形任务样本。在本文件中,我们提议戴安娜:一个动态的基于架构的终身质量保证模型,该模型试图学习一系列具有迅速增强语言模型的质量保证任务。戴安娜使用四类按等级排列的提示来捕捉不同微粒的质量保证知识。具体地说,我们专门用任务层面的提示来捕捉特定任务的知识,以保留高LL的绩效,并保持实例层面的提示,以学习不同投入样本共享的知识,改进模型的概括性工作。此外,我们专门专门专门用单独的提示来明确模拟以隐性任务和引入一套迅速的关键矢量,以促进任务之间的知识共享。广泛的实验表明,戴安娜超越了最先进的终身质量保证模型,特别是在处理无形任务方面。