Notions of fair machine learning that seek to control various kinds of error across protected groups generally are cast as constrained optimization problems over a fixed model class. For such problems, tradeoffs arise: asking for various kinds of technical fairness requires compromising on overall error, and adding more protected groups increases error rates across all groups. Our goal is to break though such accuracy-fairness tradeoffs. We develop a simple algorithmic framework that allows us to deploy models and then revise them dynamically when groups are discovered on which the error rate is suboptimal. Protected groups don't need to be pre-specified: At any point, if it is discovered that there is some group on which our current model performs substantially worse than optimally, then there is a simple update operation that improves the error on that group without increasing either overall error or the error on previously identified groups. We do not restrict the complexity of the groups that can be identified, and they can intersect in arbitrary ways. The key insight that allows us to break through the tradeoff barrier is to dynamically expand the model class as new groups are identified. The result is provably fast convergence to a model that can't be distinguished from the Bayes optimal predictor, at least by those tasked with finding high error groups. We explore two instantiations of this framework: as a "bias bug bounty" design in which external auditors are invited to discover groups on which our current model's error is suboptimal, and as an algorithmic paradigm in which the discovery of groups on which the error is suboptimal is posed as an optimization problem. In the bias bounty case, when we say that a model cannot be distinguished from Bayes optimal, we mean by any participant in the bounty program. We provide both theoretical analysis and experimental validation.
翻译:寻求控制受保护群体中各种错误的公平机器学习通常会被视为固定模型类中的限制优化问题。 对于此类问题, 出现权衡: 要求各种技术公平需要妥协总体错误, 并增加更多受保护群体, 从而增加所有群体中的错误率。 我们的目标是通过这种准确公正权衡来打破这些错误率。 我们开发一个简单的算法框架, 允许我们部署模型, 然后当发现群体出现错误率低于最优的错误时, 并动态地修改这些模型。 受保护群体不需要预先指定: 如果发现某些群体存在我们当前模式表现得比最佳要差得多的, 那么就会出现一个简单的更新操作, 在不增加总体错误或先前确定群体中的错误的情况下, 来改进该组的错误率。 我们不限制能够识别的组的复杂性, 并且它们可以任意地相互交错。 允许我们打破交易率障碍的关键洞察力是动态地扩大模式的节流周期级 。 如果发现某组, 其结果是快速接近一个模型的趋近于一个模型, 我们的模型, 也就是在其中, 我们的极值分析组中, 最高级的组中, 我们的机级分析会分析是 。