Continual learning is a challenging real-world problem for constructing a mature AI system when data are provided in a streaming fashion. Despite recent progress in continual classification, the researches of continual object detection are impeded by the diverse sizes and numbers of objects in each image. Different from previous works that tune the whole network for all tasks, in this work, we present a simple and flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTing mechAnism (ROSETTA). Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. In this way, various knowledge can be successively memorized by storing their corresponding sub-model weights in this system. To make ROSETTA automatically determine which experience is available and useful, a prototypical task correlation guided Gating Diversity Controller(GDC) is introduced to adaptively adjust the diversity of gates for the new task based on class-specific prototypes. GDC module computes class-to-class correlation matrix to depict the cross-task correlation, and hereby activates more exclusive gates for the new task if a significant domain gap is observed. Comprehensive experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance on both task-based and class-based continual object detection.
翻译:持续学习是一个挑战性的现实问题,在以流传方式提供数据时,建设成熟的AI系统是一个挑战性的现实问题。尽管最近在连续分类方面取得了进展,但持续天体探测的研究却受到每个图像中不同大小和数量物体的不同限制。与以前为所有任务调整整个网络的工程不同,在这项工作中,我们提出了一个简单灵活的框架,用于通过pRotOmodic taSk corrELLATion(ROSETTA)来持续探测物体。具体地说,所有任务都共享一个统一框架,同时引入了自动选择具体任务的次级模型,以自动选择次级模型。在这种方式中,各种知识可以通过储存相应的子模型重量来相继进行记忆。为使ROSETTA自动确定哪些经验是可用和有用的,我们引入了一种指导多样性主计长控制器(GDMC),以适应基于类别原型的新任务的门的多样性。 GDC 模块对级至级的天体与级天体相关矩阵矩阵矩阵矩阵矩阵,从而在KIT-KI的跨级测试中显示跨星级测试,并在此过程中,使CO任务更加透明地显示KIT-LILTE-C 任务显示一个高级测试的高级测试。