Optimization of discrete structures aims at generating a new structure with the better property given an existing one, which is a fundamental problem in machine learning. Different from the continuous optimization, the realistic applications of discrete optimization (e.g., text generation) are very challenging due to the complex and long-range constraints, including both syntax and semantics, in discrete structures. In this work, we present SAGS, a novel Simulated Annealing framework for Graph and Sequence optimization. The key idea is to integrate powerful neural networks into metaheuristics (e.g., simulated annealing, SA) to restrict the search space in discrete optimization. We start by defining a sophisticated objective function, involving the property of interest and pre-defined constraints (e.g., grammar validity). SAGS searches from the discrete space towards this objective by performing a sequence of local edits, where deep generative neural networks propose the editing content and thus can control the quality of editing. We evaluate SAGS on paraphrase generation and molecule generation for sequence optimization and graph optimization, respectively. Extensive results show that our approach achieves state-of-the-art performance compared with existing paraphrase generation methods in terms of both automatic and human evaluations. Further, SAGS also significantly outperforms all the previous methods in molecule generation.
翻译:优化离散结构的优化旨在产生一个新的结构,其现有属性更佳,这是机器学习中的一个根本问题。与连续优化不同,离散优化(如文本生成)的现实应用由于复杂和长期的限制,包括断裂结构中的语法和语义限制,因此非常具有挑战性。在这项工作中,我们提出SAGS,一个用于图形和序列优化的新型模拟安妮框架。关键思想是将强大的神经网络纳入美经学(如模拟肛门、SA),以限制离散优化的搜索空间。我们首先界定一个复杂的客观功能,涉及兴趣属性和预先界定的限制(如语法有效性)。SAGS,从离散空间搜索这一目标,进行一系列本地编辑,其中深色化神经网络提议编辑内容,从而能够控制编辑质量。我们评估了SAMGS, 将单调生成和分子生成用于序列优化和图形优化。我们首先界定的客观功能功能功能功能,包括兴趣属性和预先界定的限制(如语法有效性)。SAGS,从离散空间搜索这一目标,进行一系列的本地编辑,其中深色神经网络提议编辑内容,从而控制编辑质量。我们分别评估了SALD生成和分子优化顺序和图表优化。广度生成方法,进一步展示了人类现有生成方法,还显示现有生成方法,进一步实现了。