We give a general framework for inference in spanning tree models. We propose unified algorithms for the important cases of first-order expectations and second-order expectations in edge-factored, non-projective spanning-tree models. Our algorithms exploit a fundamental connection between gradients and expectations, which allows us to derive efficient algorithms. These algorithms are easy to implement with or without automatic differentiation software. We motivate the development of our framework with several \emph{cautionary tales} of previous research, which has developed numerous inefficient algorithms for computing expectations and their gradients. We demonstrate how our framework efficiently computes several quantities with known algorithms, including the expected attachment score, entropy, and generalized expectation criteria. As a bonus, we give algorithms for quantities that are missing in the literature, including the KL divergence. In all cases, our approach matches the efficiency of existing algorithms and, in several cases, reduces the runtime complexity by a factor of the sentence length. We validate the implementation of our framework through runtime experiments. We find our algorithms are up to 15 and 9 times faster than previous algorithms for computing the Shannon entropy and the gradient of the generalized expectation objective, respectively.
翻译:我们为横跨树形模型的推断提供一个总体框架。 我们为一阶期望和二阶期望的重要案例提出统一的算法。 我们的算法利用梯度和期望之间的根本联系,从而使我们能够获得高效的算法。 这些算法很容易用自动区分软件来实施。 我们用以前研究的几种计算期望和梯度的参数来激励我们的框架的开发。 我们通过运行时间实验来验证我们框架的执行情况。 我们发现我们的框架有效地用已知的算法计算了若干数量,包括预期的附加评分、英特普和普遍预期标准。 作为奖金,我们为文献中缺失的数量提供算法,包括KL差异。 在所有情况下,我们的方法都与现有算法的效率相匹配,并在一些情况下,通过句号长度的一个因素来降低运行的复杂时间。 我们通过运行时间实验来验证我们框架的执行情况。 我们发现,我们的算法比普通化和香农金色的预期值分别高出15至9倍。