Describing the relationship between the variables in a study domain and modelling the data generating mechanism is a fundamental problem in many empirical sciences. Probabilistic graphical models are one common approach to tackle the problem. Learning the graphical structure is computationally challenging and a fervent area of current research with a plethora of algorithms being developed. To facilitate the benchmarking of different methods, we present a novel automated workflow, called benchpress for producing scalable, reproducible, and platform-independent benchmarks of structure learning algorithms for probabilistic graphical models. Benchpress is interfaced via a simple JSON-file, which makes it accessible for all users, while the code is designed in a fully modular fashion to enable researchers to contribute additional methodologies. Benchpress currently provides an interface to a large number of state-of-the-art algorithms from libraries such as BDgraph, BiDAG, bnlearn, GOBNILP, pcalg, r.blip, scikit-learn, TETRAD, and trilearn as well as a variety of methods for data generating models and performance evaluation. Alongside user-defined models and randomly generated datasets, the software tool also includes a number of standard datasets and graphical models from the literature, which may be included in a benchmarking workflow. We demonstrate the applicability of this workflow for learning Bayesian networks in four typical data scenarios. The source code and documentation is publicly available from http://github.com/felixleopoldo/benchpress.
翻译:描述研究领域的变量与数据生成机制建模之间的关系是许多实验科学中的一个基本问题。 概率图形模型是解决这一问题的一种共同方法。 学习图形结构是计算上具有挑战性的,也是当前研究的一个热门领域,正在开发大量的算法。 为了便利不同方法的基准化, 我们提出了一个新的自动化工作流程, 叫做“ 推理”, 用于生成可缩放、 可复制的、 以及基于平台的概率图形模型的结构学习算法基准。 会场压通过简单的 Json 文件接口, 使所有用户都能使用, 而代码则以完全模块化的方式设计, 使研究人员能够贡献更多的方法。 书面压目前为来自BDgraph、 BiDAG、 bnlearn、 GOBNILP、 palig、 r. blip、 scikit-lebrbarn、 TETRAD, 以及 trileararrann, 以及数据生成模型和业绩评估的多种方法。 我们定义的模型和随机生成的模型和数据库中的数据序列中, 也包括了一种可公开检索的模型。