Cross-lingual named entity recognition (NER) suffers from data scarcity in the target languages, especially under zero-shot settings. Existing translate-train or knowledge distillation methods attempt to bridge the language gap, but often introduce a high level of noise. To solve this problem, consistency training methods regularize the model to be robust towards perturbations on data or hidden states. However, such methods are likely to violate the consistency hypothesis, or mainly focus on coarse-grain consistency. We propose ConNER as a novel consistency training framework for cross-lingual NER, which comprises of: (1) translation-based consistency training on unlabeled target-language data, and (2) dropoutbased consistency training on labeled source-language data. ConNER effectively leverages unlabeled target-language data and alleviates overfitting on the source language to enhance the cross-lingual adaptability. Experimental results show our ConNER achieves consistent improvement over various baseline methods.
翻译:现有翻译培训或知识蒸馏方法试图弥合语言差距,但往往引入高水平的噪音。为解决这一问题,一致性培训方法规范了该模式,以稳健地干扰数据或隐藏状态。然而,这些方法可能违反一致性假设,或主要侧重于粗麦的一致性。我们提议,Conner作为跨语言NER的新的一致性培训框架,其中包括:(1) 基于翻译的无标识目标语言数据一致性培训,和(2) 关于标签源语言数据的基于辍学的一致性培训。 ConNER有效地利用无标签目标语言数据,并减轻对源语言的过度调整,以加强跨语言的适应性。实验结果表明,我们的ConNER在各种基线方法上取得了一致的改进。