The performance of existing text style transfer models is severely limited by the non-parallel datasets on which the models are trained. In non-parallel datasets, no direct mapping exists between sentences of the source and target style; the style transfer models thus only receive weak supervision of the target sentences during training, which often leads the model to discard too much style-independent information, or utterly fail to transfer the style. In this work, we propose LaMer, a novel text style transfer framework based on large-scale language models. LaMer first mines the roughly parallel expressions in the non-parallel datasets with scene graphs, and then employs MLE training, followed by imitation learning refinement, to leverage the intrinsic parallelism within the data. On two benchmark tasks (sentiment & formality transfer) and a newly proposed challenging task (political stance transfer), our model achieves qualitative advances in transfer accuracy, content preservation, and fluency. Further empirical and human evaluations demonstrate that our model not only makes training more efficient, but also generates more readable and diverse expressions than previous models.
翻译:现有文本样式转换模型的性能受到模型所培训的非平行数据集的严重限制。在非平行数据集中,源词和目标样式之间没有直接绘图;样式转换模型因此在培训期间只能对目标句子进行薄弱的监督,这往往导致模型丢弃过多的风格独立信息,或者完全不能转换样式。在这项工作中,我们提议LaMer,一个基于大规模语言模型的新颖文本样式转换框架。LaMer首先将非平行数据集中大致平行的表达形式与场景图一起埋设,然后利用MLE培训,随后进行模仿学习改进,以利用数据中的内在平行性。关于两项基准任务(感化和形式转移)和一项新提出的具有挑战性的任务(政治姿态转移),我们的模式在传输精度、内容保存和流畅度方面取得了质的进步。进一步的实证和人类评估表明,我们的模型不仅提高了培训效率,而且比以前的模型更易读性和多样化。