Text corpora are widely used resources for measuring societal biases and stereotypes. The common approach to measuring such biases using a corpus is by calculating the similarities between the embedding vector of a word (like nurse) and the vectors of the representative words of the concepts of interest (such as genders). In this study, we show that, depending on what one aims to quantify as bias, this commonly-used approach can introduce non-relevant concepts into bias measurement. We propose an alternative approach to bias measurement utilizing the smoothed first-order co-occurrence relations between the word and the representative concept words, which we derive by reconstructing the co-occurrence estimates inherent in word embedding models. We compare these approaches by conducting several experiments on the scenario of measuring gender bias of occupational words, according to an English Wikipedia corpus. Our experiments show higher correlations of the measured gender bias with the actual gender bias statistics of the U.S. job market - on two collections and with a variety of word embedding models - using the first-order approach in comparison with the vector similarity-based approaches. The first-order approach also suggests a more severe bias towards female in a few specific occupations than the other approaches.
翻译:文本公司是用来衡量社会偏见和陈规定型观念的广泛资源。用一个字体衡量这种偏见的共同方法是计算一个字(如护士)嵌入矢量与利益概念(如性别)代表字矢量之间的相似性。在本研究中,我们显示,根据什么目的将性别偏见量化为偏见,这种通常使用的方法可以将非相关概念引入偏见计量。我们建议了一种衡量偏见的替代方法,利用单词和代表概念字词之间平滑的一阶共生关系,我们通过重建单词嵌入模型中固有的共生估计值来得出这一方法。我们比较了这些方法,根据一个英文维基百科,对衡量职业词性别偏见的情景进行了几次实验。我们的实验表明,衡量性别偏见与美国职业市场两性偏见的实际统计数字在两种收藏和各种词嵌入模式方面的相关性较高,使用第一阶方法与病媒类似方法相比较,在少数具体职业中也显示出比其他方法更严重的对女性的偏见。