Observational Causal Inference in Novel Diseases: A Case Study of COVID-19

A key issue for all observational causal inference is that it relies on an unverifiable assumption - that observed characteristics are sufficient to proxy for treatment confounding. In this paper we argue that in medical cases these conditions are more likely to be met in cases where standardized treatment guidelines do not yet exist. One example of such a situation is the emergence of a novel disease. We study the case of early COVID-19 in New York City hospitals and show that observational analysis of two important thereapeutics, anti-coagulation and steroid therapy, gives results that agree with later guidelines issued via combinations of randomized trials and other evidence. We also argue that observational causal inference cannot be applied mechanically and requires domain expertise by the analyst by showing a cautionary tale of a treatment that appears extremely promising in the data, but the result is due to a quirk of hospital policy.