We report on the results of a small crowdsourcing experiment conducted at a workshop on machine learning for segmentation held at the Danish Bio Imaging network meeting 2020. During the workshop we asked participants to manually segment mitochondria in three 2D patches. The aim of the experiment was to illustrate that manual annotations should not be seen as the ground truth, but as a reference standard that is subject to substantial variation. In this note we show how the large variation we observed in the segmentations can be reduced by removing the annotators with worst pair-wise agreement. Having removed the annotators with worst performance, we illustrate that the remaining variance is semantically meaningful and can be exploited to obtain segmentations of cell boundary and cell interior.
翻译:在2020年丹麦生物成像网络会议上,我们要求与会者在3个2D补丁中人工切分米托乔因德里亚部分。实验的目的是说明,不应将人工注解视为地面真相,而应将其视为一个参考标准,其差异很大。在本说明中,我们说明如何通过删除对口协议最差的注解器来缩小我们观察到的分解中的巨大差异。我们删除了性能最差的注解器,我们说明剩余的差异在语义上是有意义的,可以用来获取细胞边界和细胞内部的分解。