With the explosive increase of big data, training a Machine Learning (ML) model becomes a computation-intensive workload, which would take days or even weeks. Thus, model reuse has received attention in the ML community, where it is called transfer learning. Transfer learning avoids training a new model from scratch by transferring knowledge from a source task to a target task. Existing transfer learning methods mostly focus on how to improve the performance of the target task through a specific source model, but assume that the source model is given. As many source models are available, it is difficult for data scientists to select the best source model for the target task manually. Hence, how to efficiently select a suitable source model for model reuse is still an unsolved problem. In this paper, we propose SMS, an effective, efficient and flexible source model selection framework. SMS is effective even when source and target datasets have significantly different data labels, is flexible to support source models with any type of structure, and is efficient to avoid any training process. For each source model, SMS first vectorizes the samples in the target dataset into soft labels by directly applying this model to the target dataset, then uses Gaussian distributions to fit the clusters of soft labels, and finally measures its distinguishing ability using Gaussian mixture-based metric. Moreover, we present an improved SMS (I-SMS), which decreases the output number of source model. I-SMS can significantly reduce the selection time while retaining the selection performance of SMS. Extensive experiments on a range of practical model reuse workloads demonstrate the effectiveness and efficiency of SMS.
翻译:随着大数据的爆炸性增加,培训机器学习(ML)模式成为计算密集型工作量,需要数日甚至数周时间。因此,模型再利用在ML社区受到注意,称为转移学习。转移学习避免将知识从源任务转移到目标任务,从零开始培训新模式。现有的传输学习方法主要侧重于如何通过特定源模式改进目标任务的业绩,但假设提供了源模式。由于有许多源模式,数据科学家很难手工选择目标任务的最佳源模式。因此,如何有效地选择一个适合模式再利用的来源模式仍然是一个尚未解决的问题。在本文件中,我们建议SMS,一个有效、高效和灵活的源模式选择框架选择框架从零开始将知识从源任务转移到目标任务。即使源和目标数据集有显著不同的数据标签,也非常灵活地支持源模式使用任何类型的结构,并且高效地避免任何培训进程。对于每个源模式而言,SMS首先将目标数据集的样本转化为软标签。在直接应用这一模型的同时,将S-S的软性效率测试范围用于软性数据系统。我们使用S的升级能力模型,最终将S-S的升级数据流数据流数据流分配。