Pre-trained language models and other generative models have revolutionized NLP and beyond. However, these models tend to reproduce undesirable biases present in their training data. Also, they may overlook patterns that are important but challenging to capture. To address these limitations, researchers have introduced distributional control techniques. These techniques, not limited to language, allow controlling the prevalence (i.e., expectations) of any features of interest in the model's outputs. Despite their potential, the widespread adoption of these techniques has been hindered by the difficulty in adapting complex, disconnected code. Here, we present disco, an open-source Python library that brings these techniques to the broader public.
翻译:培训前语言模型和其他基因模型已经使NLP及以后发生革命性变化。然而,这些模型往往重复其培训数据中存在的不良偏差。它们也可能忽略重要但难以捕捉的模式。为解决这些局限性,研究人员采用了分配控制技术。这些技术不限于语言,能够控制模型产出中任何感兴趣的特征的流行(即预期 ) 。尽管这些技术具有潜力,但由于难以调整复杂、互不连接的代码,这些技术被广泛采用受到了阻碍。这里我们介绍迪斯科,这是一个开放源码的皮松图书馆,将这些技术带给广大公众。</s>