Algorithms are now routinely used to make consequential decisions that affect human lives. Examples include college admissions, medical interventions or law enforcement. While algorithms empower us to harness all information hidden in vast amounts of data, they may inadvertently amplify existing biases in the available datasets. This concern has sparked increasing interest in fair machine learning, which aims to quantify and mitigate algorithmic discrimination. Indeed, machine learning models should undergo intensive tests to detect algorithmic biases before being deployed at scale. In this paper, we use ideas from the theory of optimal transport to propose a statistical hypothesis test for detecting unfair classifiers. Leveraging the geometry of the feature space, the test statistic quantifies the distance of the empirical distribution supported on the test samples to the manifold of distributions that render a pre-trained classifier fair. We develop a rigorous hypothesis testing mechanism for assessing the probabilistic fairness of any pre-trained logistic classifier, and we show both theoretically as well as empirically that the proposed test is asymptotically correct. In addition, the proposed framework offers interpretability by identifying the most favorable perturbation of the data so that the given classifier becomes fair.
翻译:现在经常使用算法来做出影响人类生活的决定,例如大学招生、医疗干预或执法等。算法授权我们利用大量数据中隐藏的所有信息,但可能无意中扩大现有数据集中的现有偏差。这种关切已引起人们对公平机器学习的兴趣,其目的是量化和减轻算法歧视。事实上,机器学习模型在大规模部署之前,应进行密集测试,以发现算法偏差。在本文中,我们利用最佳运输理论的理念提出统计假设测试,以发现不公平的分类者。利用地貌空间的几何测量,测试统计数据量化了测试样品上所支持的经验分布的距离,使得经过预先训练的分类师能够公平分配。我们开发了一个严格的假设测试机制,以评估任何事先经过训练的后勤分类师的概率公平性。我们从理论上和从经验上都表明,拟议的测试是过于准确的。此外,拟议的框架提供了解释性,通过确定最有利的数据易渗透度,使给定的分类师变得公平。