The design of machine learning systems often requires trading off different objectives, for example, prediction error and energy consumption for deep neural networks (DNNs). Typically, no single design performs well in all objectives; therefore, finding Pareto-optimal designs is of interest. The search for Pareto-optimal designs involves evaluating designs in an iterative process, and the measurements are used to evaluate an acquisition function that guides the search process. However, measuring different objectives incurs different costs. For example, the cost of measuring the prediction error of DNNs is orders of magnitude higher than that of measuring the energy consumption of a pre-trained DNN, as it requires re-training the DNN. Current state-of-the-art methods do not consider this difference in objective evaluation cost, potentially incurring expensive evaluations of objective functions in the optimization process. In this paper, we develop a novel decoupled and cost-aware multi-objective optimization algorithm, we call Flexible Multi-Objective Bayesian Optimization (FlexiBO) to address this issue. FlexiBO weights the improvement of the hypervolume of the Pareto region by the measurement cost of each objective to balance the expense of collecting new information with the knowledge gained through objective evaluations, preventing us from performing expensive measurements for little to no gain. We evaluate FlexiBO on seven state-of-the-art DNNs for image recognition, natural language processing (NLP), and speech-to-text translation. Our results indicate that, given the same total experimental budget, FlexiBO discovers designs with 4.8$\%$ to 12.4$\%$ lower hypervolume error than the best method in state-of-the-art multi-objective optimization.
翻译:机器学习系统的设计往往需要权衡不同的目标,例如预测错误和深层神经网络(DNN)的能源消耗。通常,没有单一的设计在所有目标中都运行良好;因此,找到Pareto最佳设计是有意义的。寻找Pareto最佳设计需要在一个迭接进程中评价设计,而测量用于评价指导搜索过程的购置功能。然而,测量不同目标需要不同的成本。例如,测量DNN的预测错误的成本比测量预先训练的DNN的能源消耗量要高得多,因为它需要再培训DNN。目前最先进的方法并不考虑客观评价成本方面的差异,在优化过程中可能会对客观功能进行昂贵的评价。在本文中,我们开发了一个新的分解和成本认知的多目标优化算法,我们称之为弹性的多直言价超额美元Optibility(Fletible-lexal-lex)来解决这个问题。 FlexBO的总量重量比我们通过最新成本的计算,我们通过最新成本的计算,从每个高额预算的计算,从我们最高成本的计算中,从我们最高级预算的计算到最高级的计算,从高水平的计算,从我们最精确的计算到通过最高级的计算到最精确的计算,通过最高级的计算,通过最高级的计算,我们最高级的计算到最高级的计算到最高级的计算。