Infants' neurological development is heavily influenced by their motor skills. Evaluating a baby's movements is key to understanding possible risks of developmental disorders in their growth. Previous research in psychology has shown that measuring specific movements or gestures such as face touches in babies is essential to analyse how babies understand themselves and their context. This research proposes the first automatic approach that detects face touches from video recordings by tracking infants' movements and gestures. The study uses a multimodal feature fusion approach mixing spatial and temporal features and exploits skeleton tracking information to generate more than 170 aggregated features of hand, face and body. This research proposes data-driven machine learning models for the detection and classification of face touch in infants. We used cross dataset testing to evaluate our proposed models. The models achieved 87.0% accuracy in detecting face touches and 71.4% macro-average accuracy in detecting specific face touch locations with significant improvements over Zero Rule and uniform random chance baselines. Moreover, we show that when we run our model to extract face touch frequencies of a larger dataset, we can predict the development of fine motor skills during the first 5 months after birth.
翻译:婴儿神经学的发展受到其运动技能的很大影响。评估婴儿运动是了解其成长中发育障碍的可能风险的关键。以前对心理学的研究显示,测量婴儿面部触摸等特定运动或动作对于分析婴儿如何理解自己及其背景至关重要。这项研究提出了第一个自动方法,通过跟踪婴儿的动作和手势,检测从视频记录中看到面部触动。研究采用了多式联运特征混合方法,混合了空间和时间特征,利用骨骼跟踪信息生成170多个手、脸和身体综合特征。这项研究提出了检测婴儿面部触摸和分类的数据驱动机器学习模型。我们利用交叉数据集测试来评估我们提议的模型。模型在探测面部触摸时实现了87.0%的准确度,在探测特定面部触摸地点时实现了71.4%的宏观平均准确度,比Zero规则以及统一的随机概率基线有了显著改进。此外,我们显示,当我们运行模型以提取更大数据集的脸触频度时,我们可以预测出生后头5个月中精巧的机动技术的发展情况。