Recent information extraction approaches have relied on training deep neural models. However, such models can easily overfit noisy labels and suffer from performance degradation. While it is very costly to filter noisy labels in large learning resources, recent studies show that such labels take more training steps to be memorized and are more frequently forgotten than clean labels, therefore are identifiable in training. Motivated by such properties, we propose a simple co-regularization framework for entity-centric information extraction, which consists of several neural models with identical structures but different parameter initialization. These models are jointly optimized with the task-specific losses and are regularized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. Extensive experiments on two widely used but noisy benchmarks for information extraction, TACRED and CoNLL03, demonstrate the effectiveness of our framework. We release our code to the community for future research.
翻译:最近的信息提取方法依赖于对深层神经模型的培训,然而,这类模型可以很容易地过度安装噪音标签,并造成性能退化。虽然在大量学习资源中过滤噪音标签的费用非常昂贵,但最近的研究表明,这类标签采取更多的培训步骤进行记忆化,而且比清洁标签更经常被遗忘,因此在培训中可以识别。受这种特性的驱动,我们提议一个实体中心信息提取的简单共同规范化框架,由若干个神经模型组成,这些模型结构相同,但参数初始化不同。这些模型与具体任务的损失共同优化,并定期化,以根据协议损失产生类似的预测,防止过度安装噪音标签。关于信息提取的两种广泛使用但又吵闹不安的基准TACRED和CONLL03的大规模实验证明了我们框架的有效性。我们向社区发布了我们的代码,供今后研究。