Natural language often exhibits inherent hierarchical structure ingrained with complex syntax and semantics. However, most state-of-the-art deep generative models learn embeddings only in Euclidean vector space, without accounting for this structural property of language. In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations. An Adversarial Poincare Variational Autoencoder (APo-VAE) is presented, where both the prior and variational posterior of latent variables are defined over a Poincare ball via wrapped normal distributions. By adopting the primal-dual formulation of KL divergence, an adversarial learning procedure is introduced to empower robust model training. Extensive experiments in language modeling and dialog-response generation tasks demonstrate the winning effectiveness of the proposed APo-VAE model over VAEs in Euclidean latent space, thanks to its superb capabilities in capturing latent language hierarchies in hyperbolic space.
翻译:自然语言往往具有内在的等级结构,具有复杂的语法和语义。然而,大多数最先进的深层基因模型只学习在爱立底矢量空间嵌入,而没有考虑到语言的结构属性。在本文中,我们调查在超偏斜潜伏空间生成的文本,以学习连续的等级表层。介绍了一个反波音波音变化自动编码器(APO-VAE),其中通过包装正常分布,在poincare球上定义了潜伏变量的先前和变异后遗物。通过采用KL差异的原始二元配方,引入了对抗性学习程序,以增强强大的模式培训。在语言建模和对话-反应生成方面进行的广泛实验表明,拟议的APO-VAE模型在获取超立基层空间的潜值语言等级方面获得了成效。