These notes shows how to do inference on the Demographic Parity (DP) metric. Although the metric is a complex statistic involving min and max computations, we propose a smooth approximation of those functions and derive its asymptotic distribution. The limit of these approximations and their gradients converge to those of the true max and min functions, wherever they exist. More importantly, when the true max and min functions are not differentiable, the approximations still are, and they provide valid asymptotic inference everywhere in the domain. We conclude with some directions on how to compute confidence intervals for DP, how to test if it is under 0.8 (the U.S. Equal Employment Opportunity Commission fairness threshold), and how to do inference in an A/B test.
翻译:这些注释显示了如何对人口均等(DP)指标进行推论。尽管该指标是一个涉及最小和最大计算的一个复杂统计,但我们建议对这些函数进行平稳近似,并得出其无症状分布。这些近似值及其梯度的极限与真实最大和最小函数的极限相趋,无论这些最小函数存在在何处。更重要的是,当真正的最大和最小函数无法区分时,近似值仍然存在,并且它们提供了全域范围内的无症状有效推论。我们得出一些方向,说明如何计算DP的信任间隔,如何测试是否低于0.8(美国平等就业机会委员会公平门槛),以及如何在A/B测试中作出推断。