Accurate confidence interval estimation for non-centrality parameters and effect size indices

We recently proposed a robust effect size index (RESI) that is related to the non-centrality parameter of a test statistic. RESI is advantageous over common indices because (1) it is widely applicable to many types of data; (2) it can rely on a robust covariance estimate; (3) it can accommodate the existence of nuisance parameters. We provided a consistent estimator for the RESI, however, there is no established confidence interval (CI) estimation procedure for the RESI. Here, we use statistical theory and simulations to evaluate several CI estimation procedures for three estimators of the RESI. Our findings show (1) in contrast to common effect sizes, the robust estimator is consistent for the true effect size; (2) common CI procedures for effect sizes that are non-centrality parameters fail to cover the true effect size at the nominal level. Using the robust estimator along with the proposed bootstrap CI is generally accurate and applicable to conduct consistent estimation and valid inference for the RESI, especially when model assumptions may be violated. Based on the RESI, we propose a general framework for the analysis of effect size (ANOES), such that effect sizes and confidence intervals can be easily reported in an analysis of variance (ANOVA) table format for a wide range of models