Do In-Person Lectures Help? A Study of a Large Statistics Class

Over 1000 students over the course of four semesters were given the option of taking an introductory statistics class either by in-person attendance in lectures, augmented by having the recorded lectures available online, or by taking the same class but without the in-person lectures. Roughly equal numbers of students chose each option. As judged purely by scores on computer-graded exams, the all-online students did very slightly better. The causal effect of choosing only online lectures was estimated by adjusting for potential confounders, most importantly the incoming ACT math scores, using multiple regression, stabilized inverse propensity weights, and a doubly-robust method. The three methods gave nearly identical results, not far from the raw score difference. The point estimates of the causal effect of choosing only online lectures remained positive but were very small and not statistically significant. No statistically significant differences were found in preliminary comparisons of effects on females/males, U.S./non-U.S. citizens, freshmen/non-freshman, and lower-scoring/higher-scoring math ACT groups.