Inverse Problems and Data Assimilation with Connections to Machine Learning

These notes are designed with the aim of providing a clear and concise introduction to the subjects of inverse problems and data assimilation, and their inter-relations, together with citations to some of the relevant literature in this area. The first part of the notes is dedicated to studying the Bayesian framework for inverse problems. Techniques such as importance sampling and Markov chain Monte Carlo methods are introduced; these methods have the desirable property that in the limit of an infinite number of samples they reproduce the full posterior distribution. However, since it is often computationally expensive or intractable to implement these methods, especially in high-dimensional settings, techniques such as approximating the posterior by a Dirac or a Gaussian distribution are introduced and studied. The second part of the notes covers data assimilation. This refers to a particular class of inverse problems in which the unknown parameter is the initial condition (and/or state) of a dynamical system, and the data comprises partial and noisy observations of the (possibly stochastic) dynamical system. The third and final part of the notes describes various topics which blend the theory of inverse problems, data assimilation, and machine learning. It is shown that ideas from data assimilation may be used to study generic inverse problems, by introducing an artificial algorithmic time. Furthermore, whilst ideas from machine learning appear in the first two parts of the notes, the final part overviews the main ways in which machine learning is impacting on, and has the potential to impact on, both the subjects of inverse problems and data assimilation.