A key challenge of dialog systems research is to effectively and efficiently adapt to new domains. A scalable paradigm for adaptation necessitates the development of generalizable models that perform well in few-shot settings. In this paper, we focus on the intent classification problem which aims to identify user intents given utterances addressed to the dialog system. We propose two approaches for improving the generalizability of utterance classification models: (1) observers and (2) example-driven training. Prior work has shown that BERT-like models tend to attribute a significant amount of attention to the [CLS] token, which we hypothesize results in diluted representations. Observers are tokens that are not attended to, and are an alternative to the [CLS] token as a semantic representation of utterances. Example-driven training learns to classify utterances by comparing to examples, thereby using the underlying encoder as a sentence similarity model. These methods are complementary; improving the representation through observers allows the example-driven model to better measure sentence similarities. When combined, the proposed methods attain state-of-the-art results on three intent prediction datasets (\textsc{banking77}, \textsc{clinc150}, \textsc{hwu64}) in both the full data and few-shot (10 examples per intent) settings. Furthermore, we demonstrate that the proposed approach can transfer to new intents and across datasets without any additional training.
翻译:对话系统研究的关键挑战是如何有效和高效地适应新的领域。一个可扩展的适应范式需要开发在几发环境中效果良好的通用模型。在本文件中,我们侧重于意图分类问题,目的是确定用户对对话系统的评论,我们提出了两种方法来改进发音分类模式的可概括性:(1) 观察员和(2) 以实例为驱动的培训。先前的工作表明,像BERT一样的模型往往把大量注意力放在[CLS]标记上,我们用稀释的表示方式来估测其效果。观察者是没有被接受的标志,并且是作为发音语语的语义表示的替代物。由实例驱动的培训学会通过比较对发音进行分类,从而使用基本的编码作为句相似模式。这些方法是相辅相成的;通过观察员改进代表方式使得以实例驱动的模型能够更好地测量其他句义的相似性。合并起来,拟议的方法在三个意向预测数据集(\\ textc{\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\2\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\