Modeling distributions of covariates, or density estimation, is a core challenge in unsupervised learning. However, the majority of work only considers the joint distribution, which has limited relevance to practical situations. A more general and useful problem is arbitrary conditional density estimation, which aims to model any possible conditional distribution over a set of covariates, reflecting the more realistic setting of inference based on prior knowledge. We propose a novel method, Arbitrary Conditioning with Energy (ACE), that can simultaneously estimate the distribution $p(\mathbf{x}_u \mid \mathbf{x}_o)$ for all possible subsets of features $\mathbf{x}_u$ and $\mathbf{x}_o$. ACE uses an energy function to specify densities, bypassing the architectural restrictions imposed by alternative methods and the biases imposed by tractable parametric distributions. We also simplify the learning problem by only learning one-dimensional conditionals, from which more complex distributions can be recovered during inference. Empirically, we show that ACE achieves state-of-the-art for arbitrary conditional and marginal likelihood estimation and for tabular data imputation.
翻译:在未经监督的学习中,模拟共变分布或密度估计是一个核心挑战。然而,大多数工作只考虑联合分布,与实际情况的相关性有限。一个更普遍和有用的问题是任意的有条件密度估计,目的是在一组共变分布的基础上模拟任何可能的有条件分布,这反映了基于先前知识的更现实的推理环境。我们建议一种新颖的方法,即 " 与能源的任意配置 " (ACE),它能够同时估计所有可能的地物($\mathbf{x_u$和$\mathbf{x ⁇ o$)的分布。一个更普遍和有用的问题是,一个任意的有条件的密度估计,目的是在一组共变异物中对任何有条件分布进行模拟。 ACE使用一种能源功能来规定密度,绕过替代替代方法的建筑限制和可移动的参数分布所施加的偏差。我们还简化了学习问题,只是学习一维的有条件的,从中可以恢复更复杂的分布。