The videofluoroscopic swallowing study (VFSS) is a gold-standard imaging technique for assessing swallowing, but analysis and rating of VFSS recordings is time consuming and requires specialized training and expertise. Researchers have recently demonstrated that it is possible to automatically detect the pharyngeal phase of swallowing and to localize the bolus in VFSS recordings via computer vision, fostering the development of novel techniques for automatic VFSS analysis. However, training of algorithms to perform these tasks requires large amounts of annotated data that are seldom available. We demonstrate that the challenges of pharyngeal phase detection and bolus localization can be solved together using a single approach. We propose a deep-learning framework that jointly tackles pharyngeal phase detection and bolus localization in a weakly-supervised manner, requiring only the initial and final frames of the pharyngeal phase as ground truth annotations for the training. Our approach stems from the observation that bolus presence in the pharynx is the most prominent visual feature upon which to infer whether individual VFSS frames belong to the pharyngeal phase. We conducted extensive experiments with multiple convolutional neural networks (CNNs) on a dataset of 1245 bolus-level clips from 59 healthy subjects. We demonstrated that the pharyngeal phase can be detected with an F1-score higher than 0.9. Moreover, by processing the class activation maps of the CNNs, we were able to localize the bolus with promising results, obtaining correlations with ground truth trajectories higher than 0.9, without any manual annotations of bolus location used for training purposes. Once validated on a larger sample of participants with swallowing disorders, our framework will pave the way for the development of intelligent tools for VFSS analysis to support clinicians in swallowing assessment.
翻译:视频含氟吞咽研究(VFSS)是一种用于评估吞咽量的黄金标准成像技术,但VFSS记录的分析与评级耗费时间,需要专门培训和专门知识。研究人员最近表明,可以通过计算机视觉自动检测吞咽的石英阶段,将VFSS记录中的肉体本地化,促进开发VFSS自动分析的新颖技术。然而,为完成这些任务而培训算法需要大量附加说明的数据,而这种数据很少可用。我们表明,可以使用单一的方法,共同解决对VHIGES记录进行更高级检测和肉毒剂本地化的挑战。我们建议了一个深层次的学习框架,共同解决吞咽的石英阶段检测和肉质本地化,只需要开发蒸汽阶段的初始和最终框架,作为培训的地面图解支持。我们的方法源自观测到,在Pharynx的血液中存在肉毒的肉质,这是最突出的视觉特征,可以用来判断VFS-CNS图架是否属于59级的神经级的血液图象处理过程。我们在12级的轨道上,我们用一个直径级的直径直径分析,我们用直径直径的直径分析将一个直径直径直径。