Using Clinical Experts Beliefs to Compare Survival Models in Health Technology Assessment

Objectives: The aim of this paper is to contrast the retrospective and prospective use of experts beliefs in choosing between survival models in economic evaluations. Methods: The use of experts retrospective (posterior) beliefs is discussed. A process for prospectively quantifying prior beliefs about model parameters in five standard models is described. Statistical criterion for comparing models, and the interpretation and computation of model probabilities is discussed. A case study is provided. Results: Experts have little difficulty in expressing their posterior beliefs. Information criterion is an approximation to Bayesian model evidence and is based on data alone. In contrast, Bayes factors measure evidence in the data and makes use of prior information. When model averaging is of interest, there is no unique way to specify prior ignorance about model probabilities. Formulating and interpreting weights of similar models should acknowledge the dilution phenomenon such that highly correlated models are given smaller weights than those with low correlation. Conclusion: The retrospective use of experts beliefs to validate a model is potentially misleading, may not achieve its intended objective and is an inefficient use of information. Experts beliefs should be elicited prospectively as probability distributions to strengthen inferences, facilitate the choice of model, and mitigate the impact of dilution on model probabilities in situations when model averaging is of interest.