Automated analysis of optical colonoscopy (OC) video frames (to assist endoscopists during OC) is challenging due to variations in color, lighting, texture, and specular reflections. Previous methods either remove some of these variations via preprocessing (making pipelines cumbersome) or add diverse training data with annotations (but expensive and time-consuming). We present CLTS-GAN, a new deep learning model that gives fine control over color, lighting, texture, and specular reflection synthesis for OC video frames. We show that adding these colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods as well as drive next generation of OC simulators for training medical students. The code and pre-trained models for CLTS-GAN are available on Computational Endoscopy Platform GitHub (https://github.com/nadeemlab/CEP).
翻译:由于颜色、照明、纹理和视觉反射的变化,对光结肠镜(OC)视频框架的自动分析(在OC期间协助内分镜医生)具有挑战性,因为彩色、照明、纹理和视觉反射的变异。以前的方法要么通过预处理(使管道繁琐)消除其中一些变异,要么增加各种附加说明的培训数据(但费用昂贵和费时)。我们介绍了CLTS-GAN,这是对OC视频框架的颜色、照明、纹理和镜面反射合成进行精细控制的一个新的深层学习模型。我们显示,在培训数据中添加这些结肠镜特定增殖可以改进最先进的聚点检测/分解方法,并驱动下一代培训医科学生的OC模拟器。CLTS-GAN的代码和预培训模型可以在Computational Enoscopic Plab平台 GitHub(https://github.com/nadeemlab/CEP)上查到。