Catoni-style confidence sequences for heavy-tailed mean estimation

A confidence sequence (CS) is a sequence of confidence intervals that is valid at arbitrary data-dependent stopping times. These are being employed in an ever-widening scope of applications involving sequential experimentation, such as A/B testing, multi-armed bandits, off-policy evaluation, election auditing, etc. In this paper, we present three approaches to constructing a confidence sequence for the population mean, under the extremely relaxed assumption that only an upper bound on the variance is known. While previous works all rely on stringent tail-lightness assumptions like boundedness or sub-Gaussianity (under which all moments of a distribution exist), the confidence sequences in our work are able to handle data from a wide range of heavy-tailed distributions (where no moment beyond the second is required to exist). Moreover, we show that even under such a simple assumption, the best among our three methods, namely the Catoni-style confidence sequence, performs remarkably well in terms of tightness, essentially matching the best methods for sub-Gaussian data. Our findings have important practical implications when experimenting with unbounded observations, since the finite-variance assumption is often more realistic and easier to verify than sub-Gaussianity.