We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective. The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives -- both with uninformed (data-independent) and informed (data-dependent) priors.
翻译:我们调查了多数票相对于有限分类组别多数票的随机对应方,并研究了其一般化特性。虽然我们的方法是任意分配的,但我们用Drichlet的分布来反省它:这允许对预期风险采用封闭式和可区别的表达方式,然后将一般化转化为可移植的培训目标。 结果的随机多数表决学习算法实现了最先进的准确性,并从(非真空的)严格概括性界限中获益。 在一系列数字实验中,与同时尽量减少PAC-Bayes目标的竞争性算法相比,这些算法既不知情(数据依赖),又知情(数据依赖)的前科。