Divide-and-Conquer MCMC for Multivariate Binary Data

The analysis of large scale medical claims data has the potential to improve quality of care by generating insights which can be used to create tailored medical programs. In particular, the multivariate probit model can be used to investigate the correlation between multiple binary responses of interest in such data, e.g. the presence of multiple chronic conditions. Bayesian modeling is well suited to such analyses because of the automatic uncertainty quantification provided by the posterior distribution. A complicating factor is that large medical claims datasets often do not fit in memory, which renders the estimation of the posterior using traditional Markov Chain Monte Carlo (MCMC) methods computationally infeasible. To address this challenge, we extend existing divide-and-conquer MCMC algorithms to the multivariate probit model, demonstrating, via simulation, that they should be preferred over mean-field variational inference when the estimation of the latent correlation structure between binary responses is of primary interest. We apply this algorithm to a large database of de-identified Medicare Advantage claims from a single large US health insurance provider, where we find medically meaningful groupings of common chronic conditions and asses the impact of the urban-rural health gap by identifying underutilized provider specialties in rural areas.