DialoGen: Generalized Long-Range Context Representation for Dialogue Systems

Long-range context modeling is crucial to both dialogue understanding and generation. The most popular method for dialogue context representation is to concatenate the last-$k$ previous utterances. However, this method may not be ideal for conversations containing long-range dependencies as it cannot look beyond last-$k$ utterances. In this work, we propose DialoGen, a novel encoder-decoder based framework for conversational response generation with a generalized context representation that can look beyond the last-$k$ utterances. Hence the method is adaptive to conversations with long-range dependencies. The main idea of our approach is to identify and utilize the most relevant historical utterances instead of the last-$k$ utterances in chronological order. We study the effectiveness of our proposed method on both dialogue generation (open-domain) and understanding (DST) tasks. DialoGen achieves comparable performance with the state-of-the-art models on DailyDialog dataset. We also observe performance gain in existing DST models with our proposed context representation strategy on MultiWOZ dataset. We discuss the generalizability and interpretability of DialoGen and show that the relevance score of previous utterances agrees well with human cognition.