Sampling from a log-concave distribution function on $\mathbb{R}^d$ (with $d\gg 1$) is a popular problem that has wide applications. In this paper we study the application of random coordinate descent method (RCD) on the Langevin Monte Carlo (LMC) sampling method, and we find two sides of the theory: 1. The direct application of RCD on LMC does reduce the number of finite differencing approximations per iteration, but it induces a large variance error term. More iterations are then needed, and ultimately the method gains no computational advantage; 2. When variance reduction techniques (such as SAGA and SVRG) are incorporated in RCD-LMC, the variance error term is reduced. The new methods, compared to the vanilla LMC, reduce the total computational cost by $d$ folds, and achieve the optimal cost rate. We perform our investigations in both overdamped and underdamped settings.
翻译:以 $mathbb{R ⁇ d$ ($d\gg 1$) 的日志组合分配函数取样是一个广受欢迎的问题,在这份文件中,我们研究了Langevin Monte Carlo(LMC) 抽样法随机协调血统法(RCD)的应用,我们发现理论的两面:1. 刚果民盟对LMC的直接应用确实减少了每个迭代的有限差价近似值的数量,但它引起了很大的差错。随后需要更多的迭代,最终方法没有计算优势;2. 当差异减少技术(如SAGA和SVRG)被纳入刚果民盟-LMC时,差异减少差错期。与Vanilla LMC相比,新方法将总计算成本减少1美元折,并达到最佳的成本率。我们在过度和未充分铺设的环境中都进行了调查。