Developing semi-supervised task-oriented dialog (TOD) systems by leveraging unlabeled dialog data has attracted increasing interests. For semi-supervised learning of latent state TOD models, variational learning is often used, but suffers from the annoying high-variance of the gradients propagated through discrete latent variables and the drawback of indirectly optimizing the target log-likelihood. Recently, an alternative algorithm, called joint stochastic approximation (JSA), has emerged for learning discrete latent variable models with impressive performances. In this paper, we propose to apply JSA to semi-supervised learning of the latent state TOD models, which is referred to as JSA-TOD. To our knowledge, JSA-TOD represents the first work in developing JSA based semi-supervised learning of discrete latent variable conditional models for such long sequential generation problems like in TOD systems. Extensive experiments show that JSA-TOD significantly outperforms its variational learning counterpart. Remarkably, semi-supervised JSA-TOD using 20% labels performs close to the full-supervised baseline on MultiWOZ2.1.
翻译:利用未贴标签的对话框数据开发半监督的任务导向对话(TOD)系统吸引了越来越多的兴趣。对于半监督的潜在TOD模型学习,通常使用变式学习,但因通过离散的潜伏变量传播梯度的高差异以及间接优化目标日志相似性的缺陷而受到影响。最近出现了一种替代算法,称为联合随机近似值(JSA),用于学习离散的潜伏变量模型,其性能令人印象深刻。在本文件中,我们提议将JSA应用到半监督的潜在的TOD模型学习中,该模型被称为JSA-TOD。就我们的知识而言,JSA-TOD代表了开发基于JSA的半监督的离散潜在有条件模型的首项工作,这些模型针对像TOD系统这样的长期相继生成问题。广泛的实验表明,JSA-TOD大大超越了它的变异性学习对应方。值得注意的是,使用20%的近于多功能2.1全超标的半超标的JSA-TOD。