跟踪LQR LQR:零级办法及其全球趋同 (LQR with Tracking: A Zeroth-order Approach and Its Global Convergence)

from arxiv, Modified parameterization for policy structure. Revised focus to be on optimization landscape and convergence for the LQR tracking problem

There has been substantial recent progress on the theoretical understanding of model-free approaches to Linear Quadratic Regulator (LQR) problems. Much attention has been devoted to the special case when the goal is to drive the state close to a zero target. In this work, we consider the general case where the target is allowed to be arbitrary, which we refer to as the LQR tracking problem. We study the optimization landscape of this problem, and show that similar to the zero-target LQR problem, the LQR tracking problem also satisfies gradient dominance and local smoothness properties. This allows us to develop a zeroth-order policy gradient algorithm that achieves global convergence. We support our arguments with numerical simulations on a linear system.

翻译：最近,在理论上理解对线性二次曲线监管(LQR)问题的无模式方法方面取得了实质性进展。当目标是将国家推向接近零目标时,对特例给予了极大的关注。在这项工作中,我们认为目标被允许为任意目标的一般情况,我们称之为LQR跟踪问题。我们研究了这一问题的优化情况,并表明,与零目标LQR问题相似,LQR跟踪问题也满足了梯度支配地位和当地平稳性能。这使我们能够开发零级政策梯度算法,实现全球趋同。我们用线性系统的数字模拟来支持我们的论点。