Flexible control of the median of the false discovery proportion

We introduce a multiple testing method that controls the median of the proportion of false discoveries (FDP) in a flexible way. Our method only requires a vector of p-values as input and is comparable to the Benjamini-Hochberg method, which controls the mean of the FDP. Benjamini-Hochberg requires choosing the target FDP alpha before looking at the data, but our method does not. For example, if using alpha=0.05 leads to no discoveries, alpha can be increased to 0.1. We further provide mFDP-adjusted p-values, which consequently also have a post hoc interpretation. The method does not assume independence and was valid in all considered simulation scenarios. The procedure is inspired by the popular estimator of the total number of true hypotheses by Schweder, Spj{\o}tvoll and Storey. We adapt this estimator to provide a median unbiased estimator of the FDP, first assuming that a fixed rejection threshold is used. Taking this as a starting point, we proceed to construct simultaneously median unbiased estimators of the FDP. This simultaneity allows for the claimed flexibility. Our method is powerful and its time complexity is linear in the number of hypotheses, after sorting the p-values.