Generalized permutation tests

Permutation tests are an immensely popular statistical tool, used for testing hypotheses of independence between variables and other common inferential questions. When the number of observations is large, it is computationally infeasible to consider every possible permutation of the data, and it is typical to either take a random draw of permutations, or to restrict to a subgroup or subset of permutations. In this work, we extend beyond these possibilities to show how such tests can be run using any distribution over any subset of permutations, with all the previous options as a special case.