Online Class-Incremental Learning For Real-World Food Classification

Online Class-Incremental Learning (OCIL) aims to continuously learn new information from single-pass data streams to update the model and mitigate catastrophic forgetting. However, most existing OCIL methods make several assumptions, including non-overlapped classes across phases and an equal number of classes in each learning phase. This is a highly simplified view of typical real-world scenarios. In this paper, we extend OCIL to the real-world food image classification task by removing these assumptions and significantly improving the performance of existing OCIL methods. We first introduce a novel probabilistic framework to simulate realistic food data sequences in different scenarios, including strict, moderate, and open diets, as a new benchmark experiment protocol. Next, we propose a novel plug-and-play module to dynamically select relevant images during training for the model update to improve learning and forgetting performance. Our proposed module can be incorporated into existing Experience Replay (ER) methods, which store representative samples from each class into an episodic memory buffer for knowledge rehearsal. We evaluate our method on the challenging Food-101 dataset and show substantial improvements over the current OCIL methods, demonstrating great potential for lifelong learning of real-world food image classification.