This study focuses on designing and developing a mathematically based quadcopter rotational dynamics simulation framework for testing reinforcement learning (RL) algorithms in many flexible configurations. The design of the simulation framework aims to simulate both linear and nonlinear representations of a quadcopter by solving initial value problems for ordinary differential equation (ODE) systems. In addition, the simulation environment is capable of making the simulation deterministic/stochastic by adding random Gaussian noise in the forms of process and measurement noises. In order to ensure that the scope of this simulation environment is not limited only with our own RL algorithms, the simulation environment has been expanded to be compatible with the OpenAI Gym toolkit. The framework also supports multiprocessing capabilities to run simulation environments simultaneously in parallel. To test these capabilities, many state-of-the-art deep RL algorithms were trained in this simulation framework and the results were compared in detail.
翻译:这项研究的重点是设计和开发一个基于数学的四氯代二苯旋转动态模拟框架,用于在许多灵活配置中测试强化学习算法(RL),模拟框架的设计旨在通过解决普通差分方程(ODE)系统的初始价值问题,模拟框架旨在模拟四氯代二苯的线性和非线性表示方式;此外,模拟环境能够通过以过程和测量噪音的形式添加随机高斯噪音,使模拟确定性/随机噪音具有确定性;为了确保这种模拟环境的范围不仅限于我们自己的RL算法,模拟环境已经扩大,以便与OpenAI Gym工具包兼容;框架还支持同时运行模拟环境的多处理能力;为测试这些能力,许多最先进的深RL算法在这一模拟框架中接受了培训,并详细比较了结果。