Due to the cost of developing and training deep learning models from scratch, machine learning engineers have begun to reuse pre-trained models (PTMs) and fine-tune them for downstream tasks. PTM registries known as "model hubs" support engineers in distributing and reusing deep learning models. PTM packages include pre-trained weights, documentation, model architectures, datasets, and metadata. Mining the information in PTM packages will enable the discovery of engineering phenomena and tools to support software engineers. However, accessing this information is difficult - there are many PTM registries, and both the registries and the individual packages may have rate limiting for accessing the data. We present an open-source dataset, PTMTorrent, to facilitate the evaluation and understanding of PTM packages. This paper describes the creation, structure, usage, and limitations of the dataset. The dataset includes a snapshot of 5 model hubs and a total of 15,913 PTM packages. These packages are represented in a uniform data schema for cross-hub mining. We describe prior uses of this data and suggest research opportunities for mining using our dataset. The PTMTorrent dataset (v1) is available at: https://app.globus.org/file-manager?origin_id=55e17a6e-9d8f-11ed-a2a2-8383522b48d9&origin_path=%2F~%2F. Our dataset generation tools are available on GitHub: https://doi.org/10.5281/zenodo.7570357.
翻译:由于从头开始开发和训练深度学习模型的成本,机器学习工程师已经开始重复使用预训练模型(PTM)并进行微调,以进行下游任务。PTM注册表称为“模型中心”支持工程师分发和重复使用深度学习模型。PTM包括预训练的权重、文档、模型架构、数据集和元数据。挖掘PTM包中的信息将使工程师发现工程现象和工具,以支持软件工程师。然而,访问这些信息是困难的-有许多PTM注册表,而且注册表和单个包都可能有访问数据的速率限制。我们提供了一个开源数据集PTMTorrent,以促进PTM包的评估和理解。本文介绍了数据集的创建、结构、使用和限制。该数据集包括5个模型中心的快照,共计15,913个PTM包。这些包采用统一的数据架构表示,以便进行跨中心挖掘。我们描述了此数据的先前用途,并提出了使用我们数据集的挖掘研究机会。PTMTorrent数据集(v1)可在以下网址获取:https://app.globus.org/file-manager?origin_id=55e17a6e-9d8f-11ed-a2a2-8383522b48d9&origin_path=%2F~%2F。我们的数据集生成工具可在GitHub上找到:https://doi.org/10.5281/zenodo.7570357。