Hypothesis Selection is a fundamental distribution learning problem where given a comparator-class $Q=\{q_1,\ldots, q_n\}$ of distributions, and a sampling access to an unknown target distribution $p$, the goal is to output a distribution $q$ such that $\mathsf{TV}(p,q)$ is close to $opt$, where $opt = \min_i\{\mathsf{TV}(p,q_i)\}$ and $\mathsf{TV}(\cdot, \cdot)$ denotes the total-variation distance. Despite the fact that this problem has been studied since the 19th century, its complexity in terms of basic resources, such as number of samples and approximation guarantees, remains unsettled (this is discussed, e.g., in the charming book by Devroye and Lugosi `00). This is in stark contrast with other (younger) learning settings, such as PAC learning, for which these complexities are well understood. We derive an optimal $2$-approximation learning strategy for the Hypothesis Selection problem, outputting $q$ such that $\mathsf{TV}(p,q) \leq2 \cdot opt + \eps$, with a (nearly) optimal sample complexity of~$\tilde O(\log n/\epsilon^2)$. This is the first algorithm that simultaneously achieves the best approximation factor and sample complexity: previously, Bousquet, Kane, and Moran (COLT `19) gave a learner achieving the optimal $2$-approximation, but with an exponentially worse sample complexity of $\tilde O(\sqrt{n}/\epsilon^{2.5})$, and Yatracos~(Annals of Statistics `85) gave a learner with optimal sample complexity of $O(\log n /\epsilon^2)$ but with a sub-optimal approximation factor of $3$.
翻译:选择 是一个基本的分布学习问题, 给一个比较器级 $q_ 1,\ ldots, q_n $, 发行量的 q_n 美元, 以及一个未知目标分配量的抽样访问 $p$, 目标是输出一个分配量 $q 美元, 这样美元接近 $opt =\ min_ i\ maths{TV} (p, q_ i) $ 和 $ mathslickr=TV} (\ cdot, ndot, 美元) 表示统计总变异的距离。 尽管这个问题自19世纪以来一直在研究过, 其基本资源的复杂性, 如样本数量和近似保证, 仍然不解析( e.) 讨论过, 德洛威和卢戈西的书中, 这与其他学习环境( 年轻) 形成鲜明的对比, 例如PAC 学习, 这些复杂性是很清楚的。