Assessing, visualizing and improving the utility of synthetic data

A number of measures have been proposed to assess the utility of the synthetic data. These include measures based on distances between the two distributions and others based on combining the original and synthetic data and predicting the origin with a propensity score. The methods will be reviewed and compared, and relations between them illustrated. These measures are incorporated into utility modules in the \pkg{synthpop} package that include methods to visualize the results. We illustrate how to compare diffent syntheses and to diagnose which aspect of the synthetic data differs from the original. The utility functions were originally designed to be used for synthetic data objects of class synds, created by synthpop, but they can now be used to compare synthetic data created by other methods with the original records. The utility measures can be standardized by their expected Null distributions from a correct synthesis model. If they are used to evaluate other types of altered data, not generated from a model, then this standardisation can be interpreted as giving the ratio of the difference for the original to the expected stochastic error.