Recent advances in text autoencoders have significantly improved the quality of the latent space, which enables models to generate grammatical and consistent text from aggregated latent vectors. As a successful application of this property, unsupervised opinion summarization models generate a summary by decoding the aggregated latent vectors of inputs. More specifically, they perform the aggregation via simple average. However, little is known about how the vector aggregation step affects the generation quality. In this study, we revisit the commonly used simple average approach by examining the latent space and generated summaries. We found that text autoencoders tend to generate overly generic summaries from simply averaged latent vectors due to an unexpected $L_2$-norm shrinkage in the aggregated latent vectors, which we refer to as summary vector degeneration. To overcome this issue, we develop a framework Coop, which searches input combinations for the latent vector aggregation using input-output word overlap. Experimental results show that Coop successfully alleviates the summary vector degeneration issue and establishes new state-of-the-art performance on two opinion summarization benchmarks. Code is available at \url{https://github.com/megagonlabs/coop}.
翻译:文本自动编码器的最近进步大大提高了潜在空间的质量,使模型能够生成来自集合潜质矢量的语法和一致文本。作为成功应用该属性,未经监督的意见总和模型通过解码投入的集合潜在矢量生成摘要。更具体地说,它们通过简单的平均值来进行聚合。然而,关于矢量聚合步骤如何影响生成质量,我们对于矢量聚合步骤如何影响生成质量知之甚少。在本研究中,我们通过审查潜质空间和生成摘要来重新审视常用的简单平均方法。我们发现,由于在聚合潜质矢量中出现出乎意料的$L_2$-norm 缩微值,因此文本自动编码往往产生过于通用的摘要。我们称之为摘要矢量矢量的汇总。为解决这一问题,我们开发了一个框架库,利用输入-输出的单词重叠来搜索潜在矢量集合的投入组合。实验结果显示,Coop成功地缓解了矢量量摘要降解问题,并在两种意见总和合成基准上建立了新的状态。代码可在\urlgoff{gom{gast/gus.