Sequential decision making for a class of hidden Markov processes, application to medical treatment optimisation

Motivated by a medical decision making problem, this paper focuses on an impulse control problem for a class of piecewise deterministic semi-Markov processes. The process evolves deterministically between jumps and the inter-jump times have a general distribution. %that may depend on both coordinates. The discrete coordinate (e.g. global health state of the patient) is not observed, the continuous one (e.g. result of some blood measurement) is observed with noise at some (possibly scarce) observation times. The objective %in this paper is to optimally select the observation dates while controlling the process so that it remains close to a nominal value. At each visit to the medical center, a cancer patient undergoes possibly invasive analyses, and {treatment and next visit dates} are scheduled according to their result and the patient history. Frequent observations lead to a better estimation of the hidden state of the process but may be too costly for the center and/or patient. Rare observations may lead to undetected possibly lethal degradation of the patient's health. We exhibit an explicit policy close to optimality based on discretisations of the process. Construction of discretisation grids are discussed at length. The paper is illustrated with experiments on synthetic data fitted from the Intergroupe Francophone du My\'elome 2009 clinical trial.