A unifying approach on bias and variance analysis for classification

Bias and variance (B&V) decomposition is frequently used as a tool for analysing classification performance. However, the standard B&V terminologies were originally defined for the regression setting and their extensions to classification has led to several different models / definitions in the literature. Although the relation between some of these models has previously been explored, their links to the standard terminology in terms of the Bayesian statistics has not been established. In this paper, we aim to provide this missing link via employing the frameworks of Tumer & Ghosh (T&G) and James. By unifying the two approaches, we relate the classification B&V defined for the 0/1 loss to the standard B&V of the boundary distributions given for the squared error loss. The closed form relationships derived in this study provide deeper understanding of the classification performance, and their example uses on predictor design and analysis are demonstrated in two case studies.