Neural abstractive summarization has been widely studied and achieved great success with large-scale corpora. However, the considerable cost of annotating data motivates the need for learning strategies under low-resource settings. In this paper, we investigate the problems of learning summarizers with only few examples and propose corresponding methods for improvements. First, typical transfer learning methods are prone to be affected by data properties and learning objectives in the pretext tasks. Therefore, based on pretrained language models, we further present a meta learning framework to transfer few-shot learning processes from source corpora to the target corpus. Second, previous methods learn from training examples without decomposing the content and preference. The generated summaries could therefore be constrained by the preference bias in the training set, especially under low-resource settings. As such, we propose decomposing the contents and preferences during learning through the parameter modulation, which enables control over preferences during inference. Third, given a target application, specifying required preferences could be non-trivial because the preferences may be difficult to derive through observations. Therefore, we propose a novel decoding method to automatically estimate suitable preferences and generate corresponding summary candidates from the few training examples. Extensive experiments demonstrate that our methods achieve state-of-the-art performance on six diverse corpora with 30.11%/33.95%/27.51% and 26.74%/31.14%/24.48% average improvements on ROUGE-1/2/L under 10- and 100-example settings.
翻译:SPEC:为低资源自动摘要分解喜好
神经自动摘要已经得到广泛研究并在大规模语料库中获得了巨大成功,但标注数据的巨大成本使得在低资源环境下学习自动摘要成为必要。本文研究只有少量训练样例时自动摘要学习的问题,并提出相关改进方法。首先,传统的迁移学习方法容易受到预测任务中数据属性和学习目标的影响。因此,基于预训练语言模型,我们进一步提出了元学习框架,将迁移自源语料库的few-shot学习过程应用于目标语料库中。第二,先前的方法在学习时没有分解内容和喜好,因此生成的摘要可能会受到训练集中的偏好限制,尤其是在低资源情况下。因此,我们提出了通过参数调制来分解内容和喜好的方法,在推理过程中实现了喜好的控制。第三,针对目标应用,指定所需的喜好可能是不容易的,因为这些喜好可能难以通过观测得出。因此,我们提出了一种新的解码方法,自动估计出适当的喜好,并从少量训练样例中生成相应的摘要候选项。大量实验证明,我们的方法在六个不同的语料库上表现出了与最先进水平的性能,其中ROUGE-1/2/L在10个和100个样例情况下的平均改善率分别为30.11%/33.95%/27.51%和26.74%/31.14%/24.48%。