Long-Horizon Task Planning and Execution with Functional Object-Oriented Networks

Following work on joint object-action representation, functional object-oriented networks (FOON) were introduced as a knowledge representation for robots. A FOON contains symbolic (high-level) concepts useful to a robot's understanding of tasks and its environment for object-level planning. Prior to this work, little has been done to show how plans acquired from FOON can be executed by a robot, as the concepts in a FOON are too abstract for immediate execution. We propose a hierarchical task planning approach that translates a FOON graph into a PDDL-based representation of domain knowledge for task planning and execution. As a result of this process, a task plan can be acquired, which can be executed by a robot from start to end, leveraging the use of action contexts and skills as dynamic movement primitives (DMPs). We demonstrate the entire pipeline from planning to execution using CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.