Image and video analysis is often a crucial step in the study of animal behavior and kinematics. Often these analyses require that the position of one or more animal landmarks are annotated (marked) in numerous images. The process of annotating landmarks can require a significant amount of time and tedious labor, which motivates the need for algorithms that can automatically annotate landmarks. In the community of scientists that use image and video analysis to study the 3D flight of animals, there has been a trend of developing more automated approaches for annotating landmarks, yet they fall short of being generally applicable. Inspired by the success of Deep Neural Networks (DNNs) on many problems in the field of computer vision, we investigate how suitable DNNs are for accurate and automatic annotation of landmarks in video datasets representative of those collected by scientists studying animals. Our work shows, through extensive experimentation on videos of hawkmoths, that DNNs are suitable for automatic and accurate landmark localization. In particular, we show that one of our proposed DNNs is more accurate than the current best algorithm for automatic localization of landmarks on hawkmoth videos. Moreover, we demonstrate how these annotations can be used to quantitatively analyze the 3D flight of a hawkmoth. To facilitate the use of DNNs by scientists from many different fields, we provide a self contained explanation of what DNNs are, how they work, and how to apply them to other datasets using the freely available library Caffe and supplemental code that we provide.
翻译:图像和视频分析往往是动物行为和运动学研究中的一个关键步骤。这些分析往往要求在许多图像中说明一个或多个动物地标的位置。说明地标的过程需要大量的时间和烦琐的劳动,这促使需要自动说明地标的算法。在使用图像和视频分析来研究3D动物飞行的科学家群体中,出现了一种趋势,即为说明地标制定更多的自动化方法,但不能普遍适用。受深神经网络(DNN)在计算机视觉领域许多问题的成功启发,我们调查DNN如何适合准确和自动地说明能够代表研究动物的科学家所收集的视频数据集中的地标。我们的工作通过对鹰门的视频进行广泛的实验,显示DNND适合自动和准确的地标本地化。特别是,我们显示,我们提议的DNNN网站中有一个比目前关于计算机视觉领域问题的最佳算法更准确,我们如何使用自动的地标数据库进行自我分析。