Measurement and comparison of distributional shift with applications to ecology, economics, and image analysis

The concept of shift is often invoked to describe directional differences in statistical moments but has not yet been established as a property of individual distributions. In the present study, we define distributional shift (DS) as the concentration of frequencies towards the lowest discrete class and derive its measurement from the sum of cumulative frequencies. We use empirical datasets to demonstrate DS as an advantageous measure of ecological rarity and as a generalisable measure of poverty and scarcity. We then define relative distributional shift (RDS) as the difference in DS between distributions, yielding a uniquely signed (i.e., directional) measure. Using simulated random sampling, we show that RDS is closely related to measures of distance, divergence, intersection, and probabilistic scoring. We apply RDS to image analysis by demonstrating its performance in the detection of light events, changes in complex patterns, patterns within visual noise, and colour shifts. Altogether, DS is an intuitive statistical property that underpins a uniquely useful comparative measure.