Inference for Two-stage Experiments under Covariate-Adaptive Randomization

This paper studies inference in two-stage randomized experiments under covariate-adaptive randomization. Here, by a two-stage randomized experiment, we mean one in which clusters (e.g., households, schools, or graph partitions) are first randomly assigned to different levels of treated fraction, followed by random assignment of units within each treated cluster to either treatment or control based on the selected treated fraction; by covariate-adaptive randomization, we mean randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve "balance" within each stratum. We examine estimation and inference of such designs under two different asymptotic regimes, namely, "small strata" (e.g., matched-pair designs) and "large strata" (e.g., stratified block randomization). Our analysis of these two regimes enables us to study a broad range of commonly used designs from the empirical literature. We establish conditions under which our estimators are consistent and asymptotically normal and develop consistent estimators of their corresponding asymptotic variances. Combining these results establishes the asymptotic validity of tests based on these estimators. We argue that ignoring covariate information in the design stage can result in efficiency loss, and commonly used inference methods that ignore or improperly use covariate information can lead to either conservative or invalid inference. Then, we apply our results to studying optimal use of covariate information under covariate-adaptive randomization in large samples, and show that a certain generalized matched-pair design achieves minimum asymptotic variance for each proposed estimator. A simulation study and empirical application confirm the practical relevance of our theoretical results.