Parameterizations for Large-Scale Variational System Identification Using Unconstrained Optimization

This paper details how to parameterize the posterior distribution of state-space systems to generate improved optimization problems for system identification using variational inference. Three different parameterizations of the assumed state-path posterior distribution are proposed based on this representation: time-varying, steady-state, and convolution smoother; each resulting in a different parameter estimator. In contrast to existing methods for variational system identification, the proposed estimators can be implemented with unconstrained optimization methods. Furthermore, when applied to mini-batches in conjunction with stochastic optimization, the convolution-smoother formulation enables identification of large linear and nonlinear state-space systems from very large datasets. For linear systems, the method achieves the same performance as the inherently sequential prediction-error methods using an embarrassingly parallel algorithm that benefits from large speedups when computed in modern graphical processing units (GPUs). The ability of the proposed estimators to identify large models, work with large datasets split into mini-batches, and work in parallel on GPUs make them well-suited for identifying deep models for applications in systems and control.