Keywords Reinforcement LM: Improving End-to-End Response Generation in Task Oriented Dialog

In task-oriented dialogs such as MultiWoZ (Budzianowski et al., 2018), an informative and successful system response needs to include key information such as the phone number of a hotel. Therefore, we hypothesize that by asking the model to focus on generating more key quantities correctly, it can achieve better overall performance. In this paper, we propose a new training algorithm, Keywords Reinforcement Language Modeling (KRLM), that aims to use a fine-grained reward function for each token and a new per-token Reinforcement Learning procedure to help the model learn keywords generation more robustly during inference. Empirical results show that our proposed KRLM training algorithm can achieve state-of-the-art performance on the inform rate, success rate, and combined score in the MultiWoZ benchmark dataset.