The Optimality of Blocking Designs in Experiments with General Response

We consider the problem of evaluating designs for a two-arm randomized experiment for general response types under both the Neyman randomization model and population model hence generalizing the work of Kapelner et al. (2021). In the Neyman model, the only source of randomness is the treatment manipulation. Under this assumption, there is no free lunch: balanced complete randomization is minimax for estimator mean squared error. In the population model, considering the criterion where unmeasured subject covariates are averaged, we show that optimal perfect-balance designs are generally not achievable. However, in the class of all block designs, the pairwise matching design of Greevy et al. (2004) is optimal in mean squared error. When considering a tail criterion of the effect of unmeasured subject covariates in mean squared error, we prove that the common design of blocking with few blocks, i.e. order of $n^{\alpha}$ blocks for $\alpha \in (1/4,1)$, performs asymptotically optimal for all common experimental responses. Theoretical results are supported by simulations.