Recent efforts for information extraction have relied on many deep neural models. However, any such models can easily overfit noisy labels and suffer from performance degradation. While it is very costly to filter noisy labels in large learning resources, recent studies show that such labels take more training steps to be memorized and are more frequently forgotten than clean labels, therefore are identifiable in training. Motivated by such properties, we propose a simple co-regularization framework for entity-centric information extraction, which consists of several neural models with different parameter initialization. These models are jointly optimized with task-specific loss, and are regularized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. In the end, we can take any of the trained models for inference. Extensive experiments on two widely used but noisy benchmarks for information extraction, TACRED and CoNLL03, demonstrate the effectiveness of our framework.
翻译:最近的信息提取工作依赖于许多深层的神经模型,然而,任何这类模型都可以很容易地过度安装噪音标签,并造成性能退化。尽管在大量学习资源中过滤噪音标签的费用非常昂贵,但最近的研究表明,这类标签采取更多的培训步骤,以便记住,而且比清洁标签更经常被遗忘,因此在培训中可以识别。受这种特性的驱动,我们提议一个实体中心信息提取的简单共同规范化框架,由若干个具有不同参数初始化的神经模型组成。这些模型与特定任务损失共同优化,并定期化,以基于协议损失产生类似的预测,防止过度安装噪音标签。归根结底,我们可以采用任何经过培训的模型进行推断。关于信息提取两个广泛使用但又吵闹起来的基准(TACRED和ConLL03)的大规模实验展示了我们框架的有效性。