On the number of bins in a rank histogram

Rank histograms are popular tools for assessing the reliability of meteorological ensemble forecast systems. A reliable forecast system leads to a uniform rank histogram, and deviations from uniformity can indicate miscalibrations. However, the ability to identify such deviations by visual inspection of rank histogram plots crucially depends on the number of bins chosen for the histogram. If too few bins are chosen, the rank histogram is likely to miss miscalibrations; if too many are chosen, even perfectly calibrated forecast systems can yield rank histograms that do not appear uniform. In this paper we address this trade-off and propose a method for choosing the number of bins for a rank histogram. The goal of our method is to select a number of bins such that the intuitive decision whether a histogram is uniform or not is as close as possible to a formal statistical test. Our results indicate that it is often appropriate to choose fewer bins than the usual choice of ensemble size plus one, especially when the number of observations available for verification is small.