Nonresponse Bias Analysis in Longitudinal Educational Assessment Studies

Longitudinal studies are subject to nonresponse when individuals fail to provide data for entire waves or particular questions of the survey. We compare approaches to nonresponse bias analysis (NRBA) in longitudinal studies and illustrate them on the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011). Wave nonresponse with attrition yields a monotone missingness pattern, and we discuss weighting and multiple imputation (MI) approaches to NRBA for monotone patterns when the missingness mechanism is assumed missing at random (MAR). Weighting adjustments are effective when the constructed weights are correlated to the survey variable of interest. MI allows for incomplete variables to be included in the imputation model, yielding potentially less biased and more efficient estimates when the variables are predictive of the survey outcome. Multilevel models with maximum likelihood estimation and marginal models estimated using generalized estimating equations can also handle incomplete longitudinal data. We add offsets in the MI results to provide sensitivity analyses to assess missing not at random deviations from MAR. We conduct NRBA for descriptive summaries and analytic model estimates and find that in the ECLS-K:2011 application NRBA yields minor changes to the substantive conclusions. The strength of evidence about our NRBA depends on the strength of the relationship between the characteristics in the nonresponse adjustment and the key survey variables, so the key to a successful NRBA is to include strong predictors.