This paper seeks to develop a deeper understanding of the fundamental properties of neural text generations models. The study of artifacts that emerge in machine generated text as a result of modeling choices is a nascent research area. Previously, the extent and degree to which these artifacts surface in generated text has not been well studied. In the spirit of better understanding generative text models and their artifacts, we propose the new task of distinguishing which of several variants of a given model generated a piece of text, and we conduct an extensive suite of diagnostic tests to observe whether modeling choices (e.g., sampling methods, top-$k$ probabilities, model architectures, etc.) leave detectable artifacts in the text they generate. Our key finding, which is backed by a rigorous set of experiments, is that such artifacts are present and that different modeling choices can be inferred by observing the generated text alone. This suggests that neural text generators may be more sensitive to various modeling choices than previously thought.
翻译:本文试图加深对神经文本代数模型基本特性的理解。 研究机器生成的文本中因模型选择而出现的文物是一个新生的研究领域。 以前,这些文物在生成文本中表面的深度和程度没有得到很好研究。 本着更好地理解基因化文本模型及其文物的精神,我们提出新的任务,即区分某个模型的若干变种中哪一个生成了一块文字,我们进行了一系列广泛的诊断性测试,以观察模型选择(例如抽样方法、最高-千元概率、模型结构等)是否在其生成的文本中留下可探测的文物。 我们的主要发现是,这些文物存在,单凭观察生成文本就可以推断出不同的模型选择。 这意味着神经文本生成器对各种模型选择可能比先前想象的更为敏感。