Recent dialogue approaches operate by reading each word in a conversation history, and aggregating accrued dialogue information into a single state. This fixed-size vector is not expandable and must maintain a consistent format over time. Other recent approaches exploit an attention mechanism to extract useful information from past conversational utterances, but this introduces an increased computational complexity. In this work, we explore the use of the Neural Turing Machine (NTM) to provide a more permanent and flexible storage mechanism for maintaining dialogue coherence. Specifically, we introduce two separate dialogue architectures based on this NTM design. The first design features a sequence-to-sequence architecture with two separate NTM modules, one for each participant in the conversation. The second memory architecture incorporates a single NTM module, which stores parallel context information for both speakers. This second design also replaces the sequence-to-sequence architecture with a neural language model, to allow for longer context of the NTM and greater understanding of the dialogue history. We report perplexity performance for both models, and compare them to existing baselines.
翻译:最近的对话方法通过在对话历史中读取每个单词来运作,并将累积的对话信息合并成一个单一状态。这种固定尺寸的矢量无法扩展,而且必须保持一个长期的统一格式。最近的其他方法利用关注机制从过去的谈话语句中提取有用的信息,但这样做增加了计算的复杂性。在这项工作中,我们探索使用神经图解机(NTM)来提供一个更永久和灵活的存储机制,以保持对话的一致性。具体地说,我们根据NTM设计引入了两个独立的对话结构。我们根据这个NTM设计引入了两个不同的对话结构。第一个设计包含一个序列到顺序的结构,有两个单独的NTM模块,每个参与者都使用一个。第二个内存结构包含一个单一的NTM模块,为两个发言者储存平行的上下文信息。第二个设计还用一个神经语言模型来取代序列到序列结构,以便延长NTM的内涵和对对话历史的理解。我们报告两种模型的模糊性性表现,并将其与现有的基线进行比较。