The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while commonly constrained to match empirically estimated feature expectations. We seek to generalize this principle to scenarios where the empirical feature expectations cannot be computed because the model variables are only partially observed, which introduces a dependency on the learned model. Extending and generalizing the principle of latent maximum entropy, we introduce uncertain maximum entropy and describe an expectation-maximization based solution to approximately solve these problems. We show that our technique generalizes the principle of maximum entropy and latent maximum entropy and discuss a generally applicable regularization technique for adding error terms to feature expectation constraints in the event of limited data.
翻译:最大倍增率原则是一种广泛适用的技术,用于计算尽可能少信息的分配,同时通常限于与经验估计的特征预期相匹配。我们力求将这一原则推广到无法计算经验特征预期的情景中,因为模型变量只得到部分遵守,从而导致对所学模型的依赖。我们推广和普及潜在最大倍增率原则,引入不确定的最大倍增率原则,并描述基于期望的最大化解决方案,以大致解决这些问题。我们表明,我们的技术概括了最大倍增率和潜在最大倍增率原则,并讨论了普遍适用的规范化技术,以便在数据有限的情况下,在预期限制上增加错误术语。