In recent years, great success has been achieved in many tasks of natural language processing (NLP), e.g., named entity recognition (NER), especially in the high-resource language, i.e., English, thanks in part to the considerable amount of labeled resources. However, most low-resource languages do not have such an abundance of labeled data as high-resource English, leading to poor performance of NER in these low-resource languages. Inspired by knowledge transfer, we propose Converse Attention Network, or CAN in short, to improve the performance of NER in low-resource languages by leveraging the knowledge learned in pretrained high-resource English models. CAN first translates low-resource languages into high-resource English using an attention based translation module. In the process of translation, CAN obtain the attention matrices that align the two languages. Furthermore, CAN use the attention matrices to align the high-resource semantic features from a pretrained high-resource English model with the low-resource semantic features. As a result, CAN obtains aligned high-resource semantic features to enrich the representations of low-resource languages. Experiments on four low-resource NER datasets show that CAN achieves consistent and significant performance improvements, which indicates the effectiveness of CAN.
翻译:近年来,在自然语言处理(NLP)的许多任务中取得了巨大成功,例如,命名实体承认(NER),特别是在高资源语言(即英语)方面,这在一定程度上归功于大量贴有标签的资源;然而,大多数低资源语言没有如此丰富的标签数据,如高资源英语,导致这些低资源语言的净化性能差;在知识转让的启发下,我们提议反注意力网络,或简而言之,通过利用在预先培训的高资源英语模型中所学的知识,提高低资源语言净化的绩效。CAN首先将低资源语言翻译为高资源英语,使用基于关注的翻译模块。在翻译过程中,CAN能够获得与两种语言相一致的注意矩阵。此外,CAN利用关注矩阵将受过训练的高资源高资源英语模型的高资源结构特征与低资源语性特征相协调。因此,CAN获得与高资源性能特征相匹配的高资源性能特征,以丰富低资源语言的表述。