Measuring covariate balance in weighted propensity score analyses by the weighted z-difference

Propensity score (PS) methods have been increasingly used in recent years when assessing treatment effects in nonrandomized studies. In terms of statistical methods, a number of new PS weighting methods were developed, and it was shown that they can outperform PS matching in efficiency of treatment effect estimation in different simulation settings. For assessing balance of covariates in treatment groups, PS weighting methods commonly use the weighted standardized difference, despite some deficiencies of this measure like, for example, the distribution of the weighted standardized difference depending on the sample size and on the distribution of weights. We introduce the weighted z-difference as a balance measure in PS weighting analyses and demonstrate its usage in a simulation study and by applying it to an example from cardiac surgery. The weighted z-difference is computationally simple and can be calculated for continuous, binary, ordinal and nominal covariates. By using Q-Q-plots we can compare the balance of PS weighted samples immediately to the balance in perfectly matched PS samples and to the expected balance in a randomized trial.