Policy Implications of Statistical Estimates: A General Bayesian Decision-Theoretic Model for Binary Outcomes

How should we evaluate a policy's effect on the likelihood of an undesirable event, such as conflict? The conventional practice has three limitations. First, relying on statistical significance misses the fact that uncertainty is a continuous scale. Second, focusing on a standard point estimate overlooks a variation in plausible effect sizes. Third, the criterion of substantive significance is rarely explained or justified. To overcome these, my Bayesian decision-theoretic model compares the expected loss under a policy intervention with the one under no such intervention. These losses are computed as a function of a particular effect size, the probability of this effect being realized, and the ratio of the cost of an intervention to the cost of an undesirable event. The model is more practically interpretable than common statistical decision-theoretic models using the standard loss functions or the relative costs of false positives and false negatives. I exemplify my model's use through three applications and provide an R package.