Sampling from Gibbs distributions $p(x) \propto \exp(-V(x)/\varepsilon)$ and computing their log-partition function are fundamental tasks in statistics, machine learning, and statistical physics. However, while efficient algorithms are known for convex potentials $V$, the situation is much more difficult in the non-convex case, where algorithms necessarily suffer from the curse of dimensionality in the worst case. For optimization, which can be seen as a low-temperature limit of sampling, it is known that smooth functions $V$ allow faster convergence rates. Specifically, for $m$-times differentiable functions in $d$ dimensions, the optimal rate for algorithms with $n$ function evaluations is known to be $O(n^{-m/d})$, where the constant can potentially depend on $m, d$ and the function to be optimized. Hence, the curse of dimensionality can be alleviated for smooth functions at least in terms of the convergence rate. Recently, it has been shown that similarly fast rates can also be achieved with polynomial runtime $O(n^{3.5})$, where the exponent $3.5$ is independent of $m$ or $d$. Hence, it is natural to ask whether similar rates for sampling and log-partition computation are possible, and whether they can be realized in polynomial time with an exponent independent of $m$ and $d$. We show that the optimal rates for sampling and log-partition computation are sometimes equal and sometimes faster than for optimization. We then analyze various polynomial-time sampling algorithms, including an extension of a recent promising optimization approach, and find that they sometimes exhibit interesting behavior but no near-optimal rates. Our results also give further insights on the relation between sampling, log-partition, and optimization problems.
翻译:从 Gibbs 分发 $p(x)\ popto = exp (- V(x)/\ varepsilon) 中取取 $p(x)\ propto = exp(- V) = 美元) 和计算其日志共享功能是统计、机器学习和统计物理方面的基本任务。然而,虽然对 comvex 潜在价值的高效算法已知为$V美元,但在非 convex 情况下,当算法必然受到维度的诅咒时,情况就更加困难得多。因此,可以被看作是一个低温的采样(- V), 已知平流函数的美元允许更快的趋同率。 具体地说,对于以美元为单位的不同功能, 以美元为单位的采样的算法, 算法的算算算法是美元, 以美元 直线程速度表示, 直线程速度是多少 。</s>