The developmental process of embryos follows a monotonic order. An embryo can progressively cleave from one cell to multiple cells and finally transform to morula and blastocyst. For time-lapse videos of embryos, most existing developmental stage classification methods conduct per-frame predictions using an image frame at each time step. However, classification using only images suffers from overlapping between cells and imbalance between stages. Temporal information can be valuable in addressing this problem by capturing movements between neighboring frames. In this work, we propose a two-stream model for developmental stage classification. Unlike previous methods, our two-stream model accepts both temporal and image information. We develop a linear-chain conditional random field (CRF) on top of neural network features extracted from the temporal and image streams to make use of both modalities. The linear-chain CRF formulation enables tractable training of global sequential models over multiple frames while also making it possible to inject monotonic development order constraints into the learning process explicitly. We demonstrate our algorithm on two time-lapse embryo video datasets: i) mouse and ii) human embryo datasets. Our method achieves 98.1 % and 80.6 % for mouse and human embryo stage classification, respectively. Our approach will enable more profound clinical and biological studies and suggests a new direction for developmental stage classification by utilizing temporal information.
翻译:胚胎的发育过程遵循单调顺序。 胚胎可以逐渐从一个细胞向多个细胞分离,并最终转换成摩鲁拉和爆炸性。 对于胚胎的延时视频,大多数现有的发育阶段分类方法在每一个时间步骤使用一个图像框架进行每框架的预测。 但是, 仅仅使用图像的分类在细胞之间有重叠, 各个阶段之间不平衡。 时间信息对于通过捕捉相邻框架之间的移动来解决这一问题很有价值。 在这项工作中, 我们提出一个用于发展阶段分类的双流模式。 与以往的方法不同, 我们的双流模型接受时间和图像信息。 我们从时间和图像流提取的神经网络特征顶端开发一个线链有条件随机字段(CRF), 以便使用两种模式。 线链式通用报告格式的配置使得能够对多个框架的全球顺序模型进行可移动的培训, 同时能够将单调的发育顺序限制引入学习过程。 我们用两个时间折断的胚胎视频数据集展示了我们的算法: 一) 鼠标和二) 人类胚胎新数据集。 我们的方法将达到98.1%和80.6%的临床方法将分别用于临床阶段, 和80. 和人类胚胎分析。 我们的临床方法将分别利用一个阶段, 通过鼠和生物阶段, 和人类胚胎的临床分析方法将利用一个方向, 将引导, 和生物阶段, 将使得我们的临床和生物阶段, 向。