We contribute to the discussion on parsing performance in NLP by introducing a measurement that evaluates the differences between the distributions of edge displacement (the directed distance of edges) seen in training and test data. We hypothesize that this measurement will be related to differences observed in parsing performance across treebanks. We motivate this by building upon previous work and then attempt to falsify this hypothesis by using a number of statistical methods. We establish that there is a statistical correlation between this measurement and parsing performance even when controlling for potential covariants. We then use this to establish a sampling technique that gives us an adversarial and complementary split. This gives an idea of the lower and upper bounds of parsing systems for a given treebank in lieu of freshly sampled data. In a broader sense, the methodology presented here can act as a reference for future correlation-based exploratory work in NLP.
翻译:我们通过采用一种衡量方法,评估在培训和测试数据中看到的边缘偏移(边缘的定向距离)分布差异,为关于非木材厂分解绩效的讨论作出贡献。我们假设,这一衡量方法与在树库间分解绩效中观察到的差异有关。我们利用以前的工作来推动这项工作,然后试图利用一些统计方法来伪造这一假设。我们确定,即使在控制潜在的共产物时,这种测量和分解绩效之间也有统计相关性。然后,我们用这个方法来建立一个抽样技术,使我们可以进行对立和互补的分离。这让我们可以了解给定的树库分配系统的上下层和上层界限,以取代新的抽样数据。从更广泛的意义上讲,这里提出的方法可以作为今后国家木材厂内基于相关性的探索工作的参考。