Modern machine learning tasks often require considering not just one but multiple objectives. For example, besides the prediction quality, this could be the efficiency, robustness or fairness of the learned models, or any of their combinations. Multi-objective learning offers a natural framework for handling such problems without having to commit to early trade-offs. Surprisingly, statistical learning theory so far offers almost no insight into the generalization properties of multi-objective learning. In this work, we make first steps to fill this gap: we establish foundational generalization bounds for the multi-objective setting as well as generalization and excess bounds for learning with scalarizations. We also provide the first theoretical analysis of the relation between the Pareto-optimal sets of the true objectives and the Pareto-optimal sets of their empirical approximations from training data. In particular, we show a surprising asymmetry: all Pareto-optimal solutions can be approximated by empirically Pareto-optimal ones, but not vice versa.
翻译:现代机器学习任务往往不仅需要考虑一个目标,也需要考虑多个目标。例如,除了预测质量外,这可以是所学模型或任何组合的效率、稳健性或公平性。多目标学习为处理这类问题提供了一个自然框架,而不必对早期权衡作出承诺。令人惊讶的是,统计学习理论迄今几乎无法深入了解多目标学习的概括性特性。在这项工作中,我们为填补这一差距迈出了第一步:我们为多目标设置建立了基础性概括性界限,以及用缩放法和超大界限进行学习。我们还对真实目标的Pareto最佳组合与培训数据中经验性近似的Pareto最佳组合之间的关系提供了初步理论分析。特别是,我们显示了一种令人惊讶的不对称:所有Pareto-optimal解决方案都可以被经验性Pareto-opatimic 解决方案所近似,但反之亦非如此。