One of the most important factors that contribute to the success of a machine learning model is a good training objective. Training objective crucially influences the model's performance and generalization capabilities. This paper specifically focuses on graph neural network training objective for link prediction, which has not been explored in the existing literature. Here, the training objective includes, among others, a negative sampling strategy, and various hyperparameters, such as edge message ratio which controls how training edges are used. Commonly, these hyperparameters are fine-tuned by complete grid search, which is very time-consuming and model-dependent. To mitigate these limitations, we propose Adaptive Grid Search (AdaGrid), which dynamically adjusts the edge message ratio during training. It is model agnostic and highly scalable with a fully customizable computational budget. Through extensive experiments, we show that AdaGrid can boost the performance of the models up to $1.9\%$ while being nine times more time-efficient than a complete search. Overall, AdaGrid represents an effective automated algorithm for designing machine learning training objectives.
翻译:有助于机器学习模式成功的最重要因素之一是良好的培训目标。培训目标对模型的性能和一般化能力具有至关重要的影响。本文特别侧重于用于链接预测的图形神经网络培训目标,而现有文献中尚未对此加以探讨。在这里,培训目标除其他外包括负面抽样战略和各种超参数,例如控制如何使用培训边缘的边缘信息比等。通常,这些超参数通过完全的网格搜索进行微调,这种搜索非常耗时且依赖模型。为减轻这些限制,我们提议调整网格搜索(AdaGrid),以动态调整培训期间的边缘信息比值。这是模型的不可知性,而且高度可与完全可定制的计算预算相适应。我们通过广泛的实验表明,AdaGrid可以提高模型的性能,达到1.9美元,而时间效率则比全面搜索高出9倍。总的来说,AdaGrid是设计机器学习培训目标的有效自动算法。