Extensive efforts have been made to understand and improve the fairness of machine learning models based on observational metrics, especially in high-stakes domains such as medical insurance, education, and hiring decisions. However, there is a lack of certified fairness considering the end-to-end performance of an ML model. In this paper, we first formulate the certified fairness of an ML model trained on a given data distribution as an optimization problem based on the model performance loss bound on a fairness constrained distribution, which is within bounded distributional distance with the training distribution. We then propose a general fairness certification framework and instantiate it for both sensitive shifting and general shifting scenarios. In particular, we propose to solve the optimization problem by decomposing the original data distribution into analytical subpopulations and proving the convexity of the subproblems to solve them. We evaluate our certified fairness on six real-world datasets and show that our certification is tight in the sensitive shifting scenario and provides non-trivial certification under general shifting. Our framework is flexible to integrate additional non-skewness constraints and we show that it provides even tighter certification under different real-world scenarios. We also compare our certified fairness bound with adapted existing distributional robustness bounds on Gaussian data and demonstrate that our method is significantly tighter.
翻译:已作出广泛努力,以了解和改进基于观察指标的机器学习模式的公平性,特别是在医疗保险、教育和雇用决定等高考领域;然而,考虑到ML模型的端至端性能,缺乏核证的公平性;在本文件中,我们首先根据基于公平性受限分布的模型性能损失,将特定数据分配培训的ML模型的核证公平性作为优化问题,该模型在与培训分布的受约束分配距离以内;然后,我们提议一个总的公平认证框架,对敏感的变化和一般变化情况进行即时处理;特别是,我们提议解决优化问题,将原始数据分布分解成分析亚群群,并证明解决次级问题的方法的共性;我们首先根据对六个真实世界数据集的核证公平性进行评估,并表明我们的认证在敏感的变化情景中很紧凑,在总体变化中提供非三联认证。我们的框架灵活地结合了额外的非怀疑性限制,并且我们表明,在不同的现实约束情景下,我们甚至提供了更严格的认证。