Forming Human-Robot Cooperation for Tasks with General Goal using Evolutionary Value Learning

In Human-Robot Cooperation (HRC), the robot cooperates with humans to accomplish the task together. Existing approaches assume the human has a specific goal during the cooperation, and the robot infers and acts toward it. However, in real-world environments, a human usually only has a general goal (e.g., general direction or area in motion planning) at the beginning of the cooperation, which needs to be clarified to a specific goal (e.g., an exact position) during cooperation. The specification process is interactive and dynamic, which depends on the environment and the partners' behavior. The robot that does not consider the goal specification process may cause frustration to the human partner, elongate the time to come to an agreement, and compromise or fail team performance. We present the Evolutionary Value Learning (EVL) approach, which uses a State-based Multivariate Bayesian Inference method to model the dynamics of the goal specification process in HRC. EVL can actively enhance the process of goal specification and cooperation formation. This enables the robot to simultaneously help the human specify the goal and learn a cooperative policy in a Deep Reinforcement Learning (DRL) manner. In a dynamic ball balancing task with real human subjects, the robot equipped with EVL outperforms existing methods with faster goal specification processes and better team performance.