In recent years, the use of word embeddings has become popular to measure the presence of biases in texts. Despite the fact that these measures have been shown to be effective in detecting a wide variety of biases, metrics based on word embeddings lack transparency, explainability and interpretability. In this study, we propose a PMI-based metric to quantify biases in texts. We show that this metric can be approximated by an odds ratio, which allows estimating the confidence interval and statistical significance of textual bias. We also show that this PMI-based measure can be expressed as a function of conditional probabilities, providing a simple interpretation in terms of word co-occurrences. Our approach produces a performance comparable to GloVe-based and Skip-gram-based metrics in experiments of gender-occupation and gender-name associations. We discuss the advantages and disadvantages of using methods based on first-order vs second-order co-occurrences, from the point of view of the interpretability of the metric and the sparseness of the data.
翻译:近年来,采用隐含字词的方法来衡量文本中存在偏见的情况已变得很普遍。尽管这些措施在发现各种偏见方面证明是有效的,但基于隐含字数的衡量方法缺乏透明度、可解释性和可解释性。在本研究报告中,我们提议采用基于PMI的衡量方法,以量化文本中的偏见。我们表明,这一衡量方法可以用差异比对准,从而可以估计文字偏差的信任间隔和统计意义。我们还表明,这一基于PMI的措施可以表现为有条件概率的功能,对共同发生字数进行简单的解释。我们的方法在性别职业和性别名称协会实验中产生与基于GloVe和基于GVGVE的衡量方法相类似的业绩。我们从指标可解释性和数据稀少的角度,讨论使用基于一阶法的方法的利弊。我们从第一阶法和二阶法共同发生的方法的利弊。我们从该指标可解释性和数据稀少的角度,讨论使用方法的利弊。