Statistical decision problems are the foundation of statistical machine learning. The simplest problems are binary and multiclass classification and class probability estimation. Central to their definition is the choice of loss function, which is the means by which the quality of a solution is evaluated. In this paper we systematically develop the theory of loss functions for such problems from a novel perspective whose basic ingredients are convex sets with a particular structure. The loss function is defined as the subgradient of the support function of the convex set. It is consequently automatically proper (calibrated for probability estimation). This perspective provides three novel opportunities. It enables the development of a fundamental relationship between losses and (anti)-norms that appears to have not been noticed before. Second, it enables the development of a calculus of losses induced by the calculus of convex sets which allows the interpolation between different losses, and thus is a potential useful design tool for tailoring losses to particular problems. In doing this we build upon, and considerably extend, existing results on M-sums of convex sets. Third, the perspective leads to a natural theory of `polar' (or `inverse') loss functions, which are derived from the polar dual of the convex set defining the loss, and which form a natural universal substitution function for Vovk's aggregating algorithm.
翻译:统计决策问题是统计机器学习的基础。 最简单的问题在于二进制和多级分类以及分类概率估计。 其定义的核心是选择损失函数,这是评估解决办法质量的手段。 在本文件中,我们系统地从一个新角度发展这些问题的损失函数理论,其基本成分是同特定结构相交的, 将损失函数定义为Convex集支持功能的子梯级, 因此它自动地( 校准于概率估计) 。 这个视角提供了三个新的机会。 它使得损失和( ant) 规范之间能够发展一种基本的关系, 而这种联系似乎是评估解决办法质量的手段。 其次,它能够从新的角度, 系统地发展由Convex组的微积分引起的损失函数引起的损失函数的计算方法,使不同的损失能够相互交替,从而成为使损失适应特定问题的可能有用的设计工具。 在进行这项工作时,我们利用并大大扩展了M- Comvex 集组的现有结果。 第三, 视角导致“ Polx ” 和 “ antimal comlial complain ” 函数的自然理论, 、 和“ complainal deleval seal dival seal ” 函数的推导成为双重的双重式, 。