In structured prediction, the goal is to jointly predict many output variables that together encode a structured object -- a path in a graph, an entity-relation triple, or an ordering of objects. Such a large output space makes learning hard and requires vast amounts of labeled data. Different approaches leverage alternate sources of supervision. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. It extracts supervision from unlabeled examples, but remains agnostic to the structure of the output space. Conversely, neuro-symbolic approaches exploit the knowledge that not every prediction corresponds to a valid structure in the output space. Yet, they does not further restrict the learned output distribution. This paper introduces a framework that unifies both approaches. We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object. It is obtained by restricting entropy regularization to the distribution over only valid structures. This loss is efficiently computed when the output constraint is expressed as a tractable logic circuit. Moreover, it seamlessly integrates with other neuro-symbolic losses that eliminate invalid predictions. We demonstrate the efficacy of our approach on a series of semi-supervised and fully-supervised structured-prediction experiments, where we find that it leads to models whose predictions are more accurate and more likely to be valid.
翻译:在结构化预测中,目标是共同预测将结构化对象 -- -- 图表中的路径、实体关系三、或物体的顺序排列 -- -- 编码成一个结构化对象的许多产出变量。如此大的输出空间使得学习困难,需要大量标签数据。不同的方法利用了不同的监督来源。一种方法 -- -- 加密正规化 -- -- 假设决定界限应位于低概率区域。它从未标注的例子中提取监督,但仍然对输出空间的结构具有不可知性。相反,神经顺理学方法利用并非每个预测都与产出空间的有效结构相对应的知识。然而,它们并不进一步限制所学的产出分布。本文介绍了一个统一两种方法的框架。我们建议了一种损失,即神经顺理的加密加密加密,这鼓励模型可靠地预测一个有效的对象。通过将树本正规化限制在只有有效结构的分布上获得。当输出限制被表述为可感动逻辑电路时,这种损失是有效的计算。此外,它与其他神经顺理学损失的结合是无缝的,而其他神经-顺理学损失则有可能使两种方法一致地纳入,从而消除无效的预测。我们更精确的模型。我们发现一个更精确的模型。我们可以完全的预测。我们用。我们发现一个更精确的模型。我们找到一个更精确的模型。我们可以完全的模型。