The maximum likelihood method is the best-known method for estimating the probabilities behind the data. However, the conventional method obtains the probability model closest to the empirical distribution, resulting in overfitting. Then regularization methods prevent the model from being excessively close to the wrong probability, but little is known systematically about their performance. The idea of regularization is similar to error-correcting codes, which obtain optimal decoding by mixing suboptimal solutions with an incorrectly received code. The optimal decoding in error-correcting codes is achieved based on gauge symmetry. We propose a theoretically guaranteed regularization in the maximum likelihood method by focusing on a gauge symmetry in Kullback -- Leibler divergence. In our approach, we obtain the optimal model without the need to search for hyperparameters frequently appearing in regularization.
翻译:最大似然方法是估算数据概率的最知名方法。然而,传统方法获取的概率模型最接近经验分布,导致过度拟合。然后,正则化方法防止模型过于接近错误的概率,但是对它们的性能很少有系统性的了解。正则化的思想类似于纠错码,它通过将次优解与错误接收的代码混合来获得最佳解码。纠错码中的最优解码是基于规范对称性实现的。我们通过聚焦Kullback--Leibler散度中的规范对称性来提出一种理论上保证的最大似然正则化方法。在我们的方法中,我们无需经常搜索正则化中经常出现的超参数即可获得最佳模型。