The ability to identify and resolve uncertainty is crucial for the robustness of a dialogue system. Indeed, this has been confirmed empirically on systems that utilise Bayesian approaches to dialogue belief tracking. However, such systems consider only confidence estimates and have difficulty scaling to more complex settings. Neural dialogue systems, on the other hand, rarely take uncertainties into account. They are therefore overconfident in their decisions and less robust. Moreover, the performance of the tracking task is often evaluated in isolation, without consideration of its effect on the downstream policy optimisation. We propose the use of different uncertainty measures in neural belief tracking. The effects of these measures on the downstream task of policy optimisation are evaluated by adding selected measures of uncertainty to the feature space of the policy and training policies through interaction with a user simulator. Both human and simulated user results show that incorporating these measures leads to improvements both of the performance and of the robustness of the downstream dialogue policy. This highlights the importance of developing neural dialogue belief trackers that take uncertainty into account.
翻译:确定和解决不确定性的能力对于对话系统的稳健性至关重要。事实上,在使用巴伊西亚方法对对话信仰进行跟踪的系统上,已经证实了这一点。然而,这些系统只考虑信心估计,难以推广到更复杂的环境。神经对话系统很少考虑不确定性,因此,它们的决定过于自信,不够健全。此外,跟踪任务的执行情况往往被孤立地评估,而没有考虑到其对下游政策优化的影响。我们提议在神经信仰跟踪中使用不同的不确定性措施。通过与用户模拟器的互动,在政策和培训政策的特点上增加某些不确定性措施,从而评估这些措施对政策优化下游任务的影响。人类和模拟用户的结果都表明,将这些措施纳入这些措施可以改进下游对话政策的业绩和稳健性。这突出表明了发展考虑到不确定性的神经对话信仰跟踪器的重要性。