We explore the problem of selectively forgetting categories from trained CNN classification models in the federated learning (FL). Given that the data used for training cannot be accessed globally in FL, our insights probe deep into the internal influence of each channel. Through the visualization of feature maps activated by different channels, we observe that different channels have a varying contribution to different categories in image classification. Inspired by this, we propose a method for scrubbing the model clean of information about particular categories. The method does not require retraining from scratch, nor global access to the data used for training. Instead, we introduce the concept of Term Frequency Inverse Document Frequency (TF-IDF) to quantize the class discrimination of channels. Channels with high TF-IDF scores have more discrimination on the target categories and thus need to be pruned to unlearn. The channel pruning is followed by a fine-tuning process to recover the performance of the pruned model. Evaluated on CIFAR10 dataset, our method accelerates the speed of unlearning by 8.9x for the ResNet model, and 7.9x for the VGG model under no degradation in accuracy, compared to retraining from scratch. For CIFAR100 dataset, the speedups are 9.9x and 8.4x, respectively. We envision this work as a complementary block for FL towards compliance with legal and ethical criteria.
翻译:我们探讨有选择地忘记在联邦学习(FL)中经过训练的CNN分类模式中的分类的问题。鉴于在FL中无法在全球获取培训数据,我们的洞察力深入每个频道的内部影响。通过对不同频道启动的地貌图的可视化,我们发现不同频道对图像分类的不同类别有不同的贡献。受此启发,我们提出了一个清理特定类别信息的模式的方法。该方法不要求从零开始再培训,也不要求全球访问用于培训的数据。相反,我们引入了Term River Inversion Documents Rience(TF-IDF)的概念,以量化不同频道的等级歧视。高TF-IDF分数的频道在目标类别上存在更多的歧视,因此需要调整到未读的类别。受此启发,我们提出了一个精细调整过程,以恢复经调整的模型的性能。在CFFAR10数据集中,我们的方法加快了对ResNet模型的不学习速度为8.9x,而VGGI-IDS模型的7.9x,在不降解的情况下,我们用这种CRISx标准的合规性标准,我们分别与CREDRA标准为C。