We consider the infinite-horizon, discrete-time full-information control problem. Motivated by learning theory, as a criterion for controller design we focus on regret, defined as the difference between the LQR cost of a causal controller (that has only access to past and current disturbances) and the LQR cost of a clairvoyant one (that has also access to future disturbances). In the full-information setting, there is a unique optimal non-causal controller that in terms of LQR cost dominates all other controllers. Since the regret itself is a function of the disturbances, we consider the worst-case regret over all possible bounded energy disturbances, and propose to find a causal controller that minimizes this worst-case regret. The resulting controller has the interpretation of guaranteeing the smallest possible regret compared to the best non-causal controller, no matter what the future disturbances are. We show that the regret-optimal control problem can be reduced to a Nehari problem, i.e., to approximate an anticausal operator with a causal one in the operator norm. In the state-space setting, explicit formulas for the optimal regret and for the regret-optimal controller (in both the causal and the strictly causal settings) are derived. The regret-optimal controller is the sum of the classical $H_2$ state-feedback law and a finite-dimensional controller obtained from the Nehari problem. The controller construction simply requires the solution to the standard LQR Riccati equation, in addition to two Lyapunov equations. Simulations over a range of plants demonstrates that the regret-optimal controller interpolates nicely between the $H_2$ and the $H_\infty$ optimal controllers, and generally has $H_2$ and $H_\infty$ costs that are simultaneously close to their optimal values. The regret-optimal controller thus presents itself as a viable option for control system design.
翻译:我们考虑的是无限和离散的全时信息控制问题。 以学习理论为动力, 作为控制器设计的标准, 我们专注于遗憾, 被定义为因果控制者( 只能接触过去和当前的扰动) 的LQR成本与clairvoyant ( 也能够接触未来的扰动) 的LQR成本之间的差别。 在完整的信息环境中, 有一种独特的最佳非因果控制器, 以LQR 的成本控制所有其他控制器。 由于遗憾本身是扰动的函数, 我们考虑对所有可能的受绑定的液态能源扰动的最坏的遗憾, 并提议寻找一个能尽量减少这种最坏的气态控制器。 由此, 相对于最好的非因果控制器来说, 未来扰动。 我们显示, 遗憾- 最佳控制问题可以降为Nehari 问题, 也就是说, 最坏的气态操作员和最坏的液压 。 在州- horral2 和最坏的汇率中, 最明显地表示最坏的汇率和最坏的汇率。