A central goal of probabilistic programming languages (PPLs) is to separate modelling from inference. However, this goal is hard to achieve in practice. Users are often forced to re-write their models in order to improve efficiency of inference or meet restrictions imposed by the PPL. Conditional independence (CI) relationships among parameters are a crucial aspect of probabilistic models that capture a qualitative summary of the specified model and can facilitate more efficient inference. We present an information flow type system for probabilistic programming that captures conditional independence (CI) relationships, and show that, for a well-typed program in our system, the distribution it implements is guaranteed to have certain CI-relationships. Further, by using type inference, we can statically deduce which CI-properties are present in a specified model. As a practical application, we consider the problem of how to perform inference on models with mixed discrete and continuous parameters. Inference on such models is challenging in many existing PPLs, but can be improved through a workaround, where the discrete parameters are used implicitly, at the expense of manual model re-writing. We present a source-to-source semantics-preserving transformation, which uses our CI-type system to automate this workaround by eliminating the discrete parameters from a probabilistic program. The resulting program can be seen as a hybrid inference algorithm on the original program, where continuous parameters can be drawn using efficient gradient-based inference methods, while the discrete parameters are inferred using variable elimination. We implement our CI-type system and its example application in SlicStan: a compositional variant of Stan.
翻译:概率性编程语言(PPLs)的中心目标是将概率性编程与推断法区分开来。然而,这一目标在实践中很难实现。用户往往被迫重新编程模型,以提高推断效率或满足PPL规定的限制。参数之间的条件性独立性(CI)关系是概率性模型的一个关键方面,这些模型可以对特定模型进行质量总结,并能够促进更有效的推断。我们为概率性编程提供了一个信息流类型系统,可以捕捉有条件独立(CI)关系的参数,并表明,对于我们系统中的精度型程序来说,用户往往被迫重写模型,以便提高推断效率或达到PPPLs的限制。此外,通过使用类型推导,我们可以静态推论,而CI的偏差性(CI)关系是特定模型的一个关键方面。作为一个实际应用,我们考虑如何用混合和连续参数对模型进行推论的问题。关于这种模型的推论在许多现有的PPLs中具有挑战性,但可以通过一个工作周期性模型来改进,在使用离性参数的情况下,使用离性参数来进行某些分流性参数,从而将成本性程序转换。