Machine unlearning is the task of updating machine learning (ML) models after a subset of the training data they were trained on is deleted. Methods for the task are desired to combine effectiveness and efficiency, i.e., they should effectively "unlearn" deleted data, but in a way that does not require excessive computation effort (e.g., a full retraining) for a small amount of deletions. Such a combination is typically achieved by tolerating some amount of approximation in the unlearning. In addition, laws and regulations in the spirit of "the right to be forgotten" have given rise to requirements for certifiability, i.e., the ability to demonstrate that the deleted data has indeed been unlearned by the ML model. In this paper, we present an experimental study of the three state-of-the-art approximate unlearning methods for linear models and demonstrate the trade-offs between efficiency, effectiveness and certifiability offered by each method. In implementing the study, we extend some of the existing works and describe a common ML pipeline to compare and evaluate the unlearning methods on six real-world datasets and a variety of settings. We provide insights into the effect of the quantity and distribution of the deleted data on ML models and the performance of each unlearning method in different settings. We also propose a practical online strategy to determine when the accumulated error from approximate unlearning is large enough to warrant a full retrain of the ML model.
翻译:在经过培训的培训数据的一部分被删除后,机器不学习是更新机器学习(ML)模型的任务。任务的方法需要将有效性和效率结合起来,即,它们应当有效地“不阅读”删除的数据,但这样做不需要过多的计算努力(例如全面再培训),以进行少量删除。这种结合通常是通过在不学习中容忍某种程度的近似而实现的。此外,本着“被遗忘的权利”的精神,法律和规章产生了对可验证性的要求,即能够证明被删除的数据确实被ML模型所“不理解”的数据。在本文中,我们对三种最先进的线性模型的不学习方法进行了实验性研究(例如全面再培训),并展示每种方法所提供的效率、有效性和可验证性之间的利弊。在进行这项研究时,我们扩展了某些现有的工作,并描述了共同的ML管道,以比较和评价六个真实世界数据集的未学习方法,即证明ML模型确实被删除的数据确实被从M模型中汲取。我们从每个不切实际的模型中,又从一个不完全的在线分析到一个不精确的模型,我们又将提出一个在不完全的模型的模拟中,然后再分析。