The computation of anatomical information and laparoscope position is a fundamental block of robot-assisted surgical navigation in Minimally Invasive Surgery (MIS). Recovering a dense 3D structure of surgical scene using visual cues remains a challenge, and the online laparoscopic tracking mostly relies on external sensors, which increases system complexity. In this paper, we propose a learning-driven framework, in which an image-guided laparoscopic localization with 3D reconstructions of complex anatomical structures is hereby achieved. To reconstruct the 3D structure of the whole surgical environment, we first fine-tune a learning-based stereoscopic depth perception method, which is robust to the texture-less and variant soft tissues, for depth estimation. Then, we develop a dense visual reconstruction algorithm to represent the scene by surfels, estimate the laparoscope pose and fuse the depth data into a unified reference coordinate for tissue reconstruction. To estimate poses of new laparoscope views, we realize a coarse-to-fine localization method, which incorporates our reconstructed 3D model. We evaluate the reconstruction method and the localization module on three datasets, namely, the stereo correspondence and reconstruction of endoscopic data (SCARED), the ex-vivo phantom and tissue data collected with Universal Robot (UR) and Karl Storz Laparoscope, and the in-vivo DaVinci robotic surgery dataset. Extensive experiments have been conducted to prove the superior performance of our method in 3D anatomy reconstruction and laparoscopic localization, which demonstrates its potential implementation to surgical navigation system.
翻译:解剖资料和腹腔镜位置的计算是小型侵入性外科手术(MIS)中机器人辅助外科外科导航的基本块块。 使用视觉提示恢复外科手术场的密集三维结构仍是一项挑战,在线腹腔镜跟踪主要依靠外部传感器,这增加了系统的复杂性。 在本文中,我们提议了一个学习驱动框架,在这个框架中,通过3D重建复杂的解剖结构,实现图像引导腹腔镜定位。要重建整个外科环境的3D结构,我们首先精细调整一个基于学习的立体深度感知方法,该方法对无纹和变形软组织是强有力的。 然后,我们开发了一个密集的视觉重建算法,以通过冲浪机代表场,估计腹腔镜的姿势,将深度数据整合到组织重建的统一参考协调中。 我们实现了一个将3D型外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科的外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科外科