Development of COVID-19 Booster Vaccine Policy by Microsimulation and Q-learning

The COVID-19 pandemic highlighted the urgent need for effective vaccine policies, but traditional clinical trials often lack sufficient data to capture the diverse population characteristics necessary for comprehensive public health strategies. Ethical concerns around randomized trials during a pandemic further complicate policy development for public health. Reinforcement Learning (RL) offers a promising alternative for vaccine policy development. However, direct online RL exploration in real-world scenarios can result in suboptimal and potentially harmful decisions. This study proposes a novel framework combining tabular Q-learning with microsimulation (i.e., a Recurrent Neural Network (RNN) environment simulator) to address these challenges in public health vaccine policymaking, which enables effective vaccine policy learning without real-world interaction, addressing both ethical and exploration challenges. The RNN environment simulator captures temporal associations between infection and patient characteristics, generating realistic simulation data. Our tabular Q-learning model produces an interpretable policy table that balances the risks of severe infection against vaccination side effects. Applied to COVID-19 booster policies, the learned Q-learning-based policy outperforms current practices, offering a path toward more effective vaccination strategies.