In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant is normally required to control the position and zooming ratio of the laparoscope, following the surgeon's instructions. However, moving the laparoscope frequently may lead to unstable and suboptimal views, while the adjustment of zooming ratio may interrupt the workflow of the surgical operation. To this end, we propose a multi-scale Generative Adversarial Network (GAN)-based video super-resolution method to construct a framework for automatic zooming ratio adjustment. It can provide automatic real-time zooming for high-quality visualization of the Region Of Interest (ROI) during the surgical operation. In the pipeline of the framework, the Kernel Correlation Filter (KCF) tracker is used for tracking the tips of the surgical tools, while the Semi-Global Block Matching (SGBM) based depth estimation and Recurrent Neural Network (RNN)-based context-awareness are developed to determine the upscaling ratio for zooming. The framework is validated with the JIGSAW dataset and Hamlyn Centre Laparoscopic/Endoscopic Video Datasets, with results demonstrating its practicability.
翻译:在机器人辅助小型侵入外科手术中,通常需要摄像助理按照外科医生的指示,控制腹腔镜的位置和放大比例。然而,移动腹腔镜往往会导致不稳和不优化的视图,而缩放比例的调整可能会干扰外科手术的工作流程。为此,我们提议采用一个多尺度的放大反反向网络(GAN)基于视频的超分辨率方法,以构建自动缩放比率调整框架。它可以在外科手术期间为利益区高品质视觉化提供自动实时放大率。在框架的管道中,使用内核互连过滤器跟踪器跟踪外科手术工具的提示。与此同时,我们开发了基于深度估计的半全球边距匹配网络和基于常规神经网络(RNN)的环境意识,以确定缩放率的升级率。这个框架与JIGSAAW数据设置和Hamlyn光学中心验证了框架,并演示其可摄像性数据库/图像中心。