This paper explores continuous-time control synthesis for target-driven navigation to satisfy complex high-level tasks expressed as linear temporal logic (LTL). We propose a model-free framework using deep reinforcement learning (DRL) where the underlying dynamic system is unknown (an opaque box). Unlike prior work, this paper considers scenarios where the given LTL specification might be infeasible and therefore cannot be accomplished globally. Instead of modifying the given LTL formula, we provide a general DRL-based approach to satisfy it with minimal violation. %\mminline{Need to decide if we're comfortable calling these "guarantees" due to the stochastic policy. I'm not repeating this comment everywhere that says "guarantees" but there are multiple places.} To do this, we transform a previously multi-objective DRL problem, which requires simultaneous automata satisfaction and minimum violation cost, into a single objective. By guiding the DRL agent with a sampling-based path planning algorithm for the potentially infeasible LTL task, the proposed approach mitigates the myopic tendencies of DRL, which are often an issue when learning general LTL tasks that can have long or infinite horizons. This is achieved by decomposing an infeasible LTL formula into several reach-avoid sub-tasks with shorter horizons, which can be trained in a modular DRL architecture. Furthermore, we overcome the challenge of the exploration process for DRL in complex and cluttered environments by using path planners to design rewards that are dense in the configuration space. The benefits of the presented approach are demonstrated through testing on various complex nonlinear systems and compared with state-of-the-art baselines. The Video demonstration can be found on YouTube Channel:\url{https://youtu.be/jBhx6Nv224E}.
翻译:本文探索目标驱动导航的连续时间控制合成, 以满足以线性时间逻辑( LTL) 表达的复杂高级任务。 我们提议了一个使用深强化学习( DRL) 且基础动态系统未知( 一个不透明的框 ) 的无模型框架。 与先前的工作不同, 本文件考虑的是, 给定的 LTL 规格可能不可行, 因而无法在全球范围完成的情景。 我们不是修改给定的 LTL 公式, 而是提供一个通用的 DRL 方法, 以最小的违反方式满足它。 ⁇ mminline{ { 需要确定我们是否满足这些以线性时间线性时间值表示的复杂高级任务 。 提议的方法可以减轻这些“ 保证” ( DRL ) 的“ 保证 保证 ” ( DRL ) 路径值, 并且通过常规的 IMDL, 和 IML 的“ 常规 ”, 能够通过常规的 IMDL, 和 解算地 。