Hamiltonian Monte Carlo in Inverse Problems; Ill-Conditioning and Multi-Modality

The Hamiltonian Monte Carlo (HMC) method allows sampling from continuous densities. Favorable scaling with dimension has led to wide adoption of HMC by the statistics community. Modern auto-differentiating software should allow more widespread usage in Bayesian inverse problems. This paper analyzes the two major difficulties encountered using HMC for inverse problems: poor conditioning and multi-modality. Novel results on preconditioning and replica exchange Monte Carlo parameter selection are presented in the context of spectroscopy. Recommendations are analyzed rigorously in the Gaussian case, and shown to generalize in a fusion plasma reconstruction.