In assessing prediction accuracy of multivariable prediction models, optimism corrections are essential for preventing biased results. However, in most published papers of clinical prediction models, the point estimates of the prediction accuracy measures are corrected by adequate bootstrap-based correction methods, but their confidence intervals are not corrected, e.g., the DeLong's confidence interval is usually used for assessing the C-statistic. These naive methods do not adjust for the optimism bias and do not account for statistical variability in the estimation of parameters in the prediction models. Therefore, their coverage probabilities of the true value of the prediction accuracy measure can be seriously below the nominal level (e.g., 95%). In this article, we provide two generic bootstrap methods, namely (1) location-shifted bootstrap confidence intervals and (2) two-stage bootstrap confidence intervals, that can be generally applied to the bootstrap-based optimism correction methods, i.e., the Harrell's bias correction, 0.632, and 0.632+ methods. In addition, they can be widely applied to various methods for prediction model development involving modern shrinkage methods such as the ridge and lasso regressions. Through numerical evaluations by simulations, the proposed confidence intervals showed favourable coverage performances. Besides, the current standard practices based on the optimism-uncorrected methods showed serious undercoverage properties. To avoid erroneous results, the optimism-uncorrected confidence intervals should not be used in practice, and the adjusted methods are recommended instead. We also developed the R package predboot for implementing these methods (https://github.com/nomahi/predboot). The effectiveness of the proposed methods are illustrated via applications to the GUSTO-I clinical trial.
翻译:在评估多变预测模型的预测准确性时,乐观性修正对于防止有偏差的结果至关重要。然而,在大多数发表的临床预测模型论文中,预测准确度措施的点估计值通过适当的靴子式修正方法得到纠正,但其信任度间隔没有得到纠正,例如,DeLong的置信度间隔通常用于评估C-统计模型。这些天真的方法不适应乐观偏差,也不考虑预测模型参数估计的统计差异。因此,在多数发表的临床预测准确度计量的真实值的覆盖率可能大大低于名义值(例如,95%)。在本篇文章中,我们提供了两种通用的靴子测准确度方法,即:(1) 定位式靴子式信任间隔间隔间隔和(2) 两阶段的测信度间隔,通常用于评估基于靴式的乐观性修正方法,即Harrellell的偏差纠正方法,0.632和0.632+方法。此外,可以广泛应用这些方法来预测模型的模型开发,包括现代缩略式方法,如Rridge值和lassobroad road ;通过模拟评估,采用这些评估方法,以模拟方式显示以稳性分析性分析性分析性做法。采用这些方法,以模拟分析性分析性分析性做法,采用这些方法,采用这些方法,以显示以模拟性分析性平差分析性平价比。