Expected risk minimization (ERM) is at the core of many machine learning systems. This means that the risk inherent in a loss distribution is summarized using a single number - its average. In this paper, we propose a general approach to construct risk measures which exhibit a desired tail sensitivity and may replace the expectation operator in ERM. Our method relies on the specification of a reference distribution with a desired tail behaviour, which is in a one-to-one correspondence to a coherent upper probability. Any risk measure, which is compatible with this upper probability, displays a tail sensitivity which is finely tuned to the reference distribution. As a concrete example, we focus on divergence risk measures based on f-divergence ambiguity sets, which are a widespread tool used to foster distributional robustness of machine learning systems. For instance, we show how ambiguity sets based on the Kullback-Leibler divergence are intricately tied to the class of subexponential random variables. We elaborate the connection of divergence risk measures and rearrangement invariant Banach norms.
翻译:预期风险最小化(ERM)是许多机器学习系统的核心。 这意味着损失分布所固有的风险将使用一个单一数字(其平均值)来总结。 在本文中,我们提出了一个总体方法来构建风险衡量标准,这些衡量标准显示出一种理想尾部敏感度,并可能取代机构风险管理中的预期操作者。我们的方法依赖于参考分布的规格和一种理想尾部行为,这种尾部行为以一对一的对应方式与一致的上概率。任何与这一高概率兼容的风险评估措施都显示一种尾部敏感度,它与参考分布相匹配。具体地说,我们侧重于基于f-diverence模糊度的差分风险衡量标准,这是用于促进机器学习系统的分布稳健性的广泛工具。例如,我们展示了基于 Kullback-Leiber 差异的模糊度是如何与亚扩展随机变量的类别紧密联系在一起的。我们详细描述了差异风险计量和变换 Banach 规范之间的关联。