Learning Collision-free and Torque-limited Robot Trajectories based on Alternative Safe Behaviors

This paper presents an approach to learn online generation of collision-free and torque-limited robot trajectories. In order to generate future motions, a neural network is periodically invoked. Based on the current kinematic state of the robot and the network prediction, a trajectory for the current time interval can be calculated. The main idea of our paper is to execute the predicted motion only if a collision-free and torque-limited way to continue the trajectory is known. In practice, the motion predicted for the current time interval is extended by a braking trajectory and simulated using a physics engine. If the simulated trajectory complies with all safety constraints, the predicted motion is carried out. Otherwise, the braking trajectory calculated in the previous time interval serves as an alternative safe behavior. Given a task-specific reward function, the neural network is trained using reinforcement learning. The design of the action space used for reinforcement learning ensures that all predicted trajectories comply with kinematic joint limits. For our evaluation, simulated industrial robots and humanoid robots are trained to reach as many randomly placed target points as possible. We show that our method reliably prevents collisions with static obstacles and collisions between the robot arms, while generating motions that respect both torque limits and kinematic joint limits. Experiments with a real robot demonstrate that safe trajectories can be generated in real-time.