End-to-end spoken language understanding (SLU) has recently attracted increasing interest. Compared to the conventional tandem-based approach that combines speech recognition and language understanding as separate modules, the new approach extracts users' intentions directly from the speech signals, resulting in joint optimization and low latency. Such an approach, however, is typically designed to process one intention at a time, which leads users to take multiple rounds to fulfill their requirements while interacting with a dialogue system. In this paper, we propose a streaming end-to-end framework that can process multiple intentions in an online and incremental way. The backbone of our framework is a unidirectional RNN trained with the connectionist temporal classification (CTC) criterion. By this design, an intention can be identified when sufficient evidence has been accumulated, and multiple intentions can be identified sequentially. We evaluate our solution on the Fluent Speech Commands (FSC) dataset and the intent detection accuracy is about 97 % on all multi-intent settings. This result is comparable to the performance of the state-of-the-art non-streaming models, but is achieved in an online and incremental way. We also employ our model to a keyword spotting task using the Google Speech Commands dataset and the results are also highly promising.
翻译:端到端口语理解(SLU)最近引起了越来越多的兴趣。 与将语音识别和语言理解作为单独模块的常规同步方法相比,新方法直接从语音信号中提取用户的意图,导致联合优化和低潜伏。然而,这种方法通常旨在一次处理一个意图,使用户在与对话系统互动时采取多轮来满足其要求。在本文件中,我们提议了一个流式端到端框架,可以在线和渐进方式处理多种意图。我们框架的骨干是一个单向式 RNN 标准,在连接器时间分类(CTC)标准方面受过单向性 RNN 培训。通过这一设计,在收集足够证据后可以确定一种意图,并可以按顺序确定多种意图。我们评估了我们关于流式语音指令数据集的解决方案,在所有多功能环境中检测的准确度约为97%。这与状态非流动模式的性能相当,但是在在线和递增式指令中实现了。我们还利用高亮的服务器任务。