Tuning complex machine learning systems is challenging. Machine learning typically requires to set hyperparameters, be it regularization, architecture, or optimization parameters, whose tuning is critical to achieve good predictive performance. To democratize access to machine learning systems, it is essential to automate the tuning. This paper presents Amazon SageMaker Automatic Model Tuning (AMT), a fully managed system for gradient-free optimization at scale. AMT finds the best version of a trained machine learning model by repeatedly evaluating it with different hyperparameter configurations. It leverages either random search or Bayesian optimization to choose the hyperparameter values resulting in the best model, as measured by the metric chosen by the user. AMT can be used with built-in algorithms, custom algorithms, and Amazon SageMaker pre-built containers for machine learning frameworks. We discuss the core functionality, system architecture, our design principles, and lessons learned. We also describe more advanced features of AMT, such as automated early stopping and warm-starting, showing in experiments their benefits to users.
翻译:复杂的机器学习系统具有挑战性。 机器学习通常需要设置超参数, 无论是正规化、 建筑或优化参数, 其调试对于实现良好的预测性能至关重要。 使机器学习系统进入民主化, 使调试自动化至关重要 。 本文展示了亚马逊- 塞格- 马克自动调试系统( AMT ), 这是一个全面管理的大规模无梯度优化系统 。 AMT 以不同的超参数配置反复评估, 找到了经过培训的机器学习模型的最佳版本 。 它利用随机搜索或巴耶斯优化来选择产生最佳模型的超参数值, 从而产生由用户选择的量度来衡量的最佳模型 。 AMT 可用于机器学习框架的内置算法、 定制算法 和亚马逊- 塞克尔 预建的容器 。 我们讨论了核心功能、 系统架构、 我们的设计原理和经验教训。 我们还描述了亚马特系统更先进的特征, 如自动的早期停止和温暖启动, 在实验中向用户展示其好处 。