The performance of natural language generation systems has improved substantially with modern neural networks. At test time they typically employ beam search to avoid locally optimal but globally suboptimal predictions. However, due to model errors, a larger beam size can lead to deteriorating performance according to the evaluation metric. For this reason, it is common to rerank the output of beam search, but this relies on beam search to produce a good set of hypotheses, which limits the potential gains. Other alternatives to beam search require changes to the training of the model, which restricts their applicability compared to beam search. This paper proposes incremental beam manipulation, i.e. reranking the hypotheses in the beam during decoding instead of only at the end. This way, hypotheses that are unlikely to lead to a good final output are discarded, and in their place hypotheses that would have been ignored will be considered instead. Applying incremental beam manipulation leads to an improvement of 1.93 and 5.82 BLEU points over vanilla beam search for the test sets of the E2E and WebNLG challenges respectively. The proposed method also outperformed a strong reranker by 1.04 BLEU points on the E2E challenge, while being on par with it on the WebNLG dataset.
翻译:自然语言生成系统的性能随着现代神经网络而有了很大的改善。 在测试时, 自然语言生成系统的性能随着现代神经网络有了很大的改善。 在测试时, 它们通常使用波束搜索来避免局部最佳但全球次最佳的预测。 但是, 由于模型错误, 更大的波束尺寸可能会导致根据评价指标的性能恶化。 因此, 通常的做法是重新排序波束搜索的输出, 但是, 这依赖于光束搜索来产生一套好的假设, 从而限制潜在的收益。 其它的波束搜索方法需要改变模型的培训, 从而限制其应用性能, 从而限制其与波束搜索相比的应用性。 本文建议对波束操作进行增量操作, 也就是说, 在拆译过程中, 将假体的假体重新排在波束中, 而不是在结尾处重新排出。 这样, 不太可能导致好最后输出结果的假体, 而在它们的位置上, 应用增量波纹波纹操纵导致1.93 和 5.82 BLEU 的点的改进。 本文件提议的方法是搜索E2 和WebLG 挑战 。