Optimization problems involving sequential decisions in a stochastic environment were studied in Stochastic Programming (SP), Stochastic Optimal Control (SOC) and Markov Decision Processes (MDP). In this paper we mainly concentrate on SP and SOC modelling approaches. In these frameworks there are natural situations when the considered problems are convex. Classical approach to sequential optimization is based on dynamic programming. It has the problem of the so-called ``Curse of Dimensionality", in that its computational complexity increases exponentially with increase of dimension of state variables. Recent progress in solving convex multistage stochastic problems is based on cutting planes approximations of the cost-to-go (value) functions of dynamic programming equations. Cutting planes type algorithms in dynamical settings is one of the main topics of this paper. We also discuss Stochastic Approximation type methods applied to multistage stochastic optimization problems. From the computational complexity point of view, these two types of methods seem to be complimentary to each other. Cutting plane type methods can handle multistage problems with a large number of stages, but a relatively smaller number of state (decision) variables. On the other hand, stochastic approximation type methods can only deal with a small number of stages, but a large number of decision variables.
翻译:涉及随机环境下顺序决策的优化问题已在随机规划(SP),随机最优控制(SOC)和Markov决策过程(MDP)中研究。在本文中,我们主要集中于SP和SOC建模方法。在这些框架中,当所考虑的问题是凸的时,自然存在凸优化问题。顺序优化的经典方法基于动态规划。它存在所谓的“维度灾难”问题,即其计算复杂性随状态变量维数的增加呈指数级增加。解决凸多级随机问题上的最新进展基于动态规划方程中成本的割平面逼近。该文主要讨论在动态环境下的割平面类型算法。我们还讨论了应用于多级随机优化问题的随机逼近类型方法。计算复杂性方面,这两种方法似乎是互补的。割平面类型方法可以处理具有大量级别但相对较少状态(决策)变量的多级别问题。另一方面,随机逼近类型的方法只能处理级数较少但决策变量较多的问题。