Purpose: Tracking the 3D motion of the surgical tool and the patient anatomy is a fundamental requirement for computer-assisted skull-base surgery. The estimated motion can be used both for intra-operative guidance and for downstream skill analysis. Recovering such motion solely from surgical videos is desirable, as it is compliant with current clinical workflows and instrumentation. Methods: We present Tracker of Anatomy and Tool (TAToo). TAToo jointly tracks the rigid 3D motion of patient skull and surgical drill from stereo microscopic videos. TAToo estimates motion via an iterative optimization process in an end-to-end differentiable form. For robust tracking performance, TAToo adopts a probabilistic formulation and enforces geometric constraints on the object level. Results: We validate TAToo on both simulation data, where ground truth motion is available, as well as on anthropomorphic phantom data, where optical tracking provides a strong baseline. We report sub-millimeter and millimeter inter-frame tracking accuracy for skull and drill, respectively, with rotation errors below 1{\deg}. We further illustrate how TAToo may be used in a surgical navigation setting. Conclusion: We present TAToo, which simultaneously tracks the surgical tool and the patient anatomy in skull-base surgery. TAToo directly predicts the motion from surgical videos, without the need of any markers. Our results show that the performance of TAToo compares favorably to competing approaches. Future work will include fine-tuning of our depth network to reach a 1 mm clinical accuracy goal desired for surgical applications in the skull base.
翻译:目的: 跟踪手术工具和病人解剖工具的3D运动是计算机辅助头骨基础外科手术的基本要求。 估计运动可以用于操作内部指导和下游技能分析。 仅从外科视频中回收这种运动是可取的,因为它符合目前的临床工作流程和仪器。 方法: 我们提供解剖和工具的跟踪器(TaToo) 。 TAToo 联合从立体微缩视频中跟踪病人头骨和外科手术操的硬3D运动。 TAToo 以端到端不同的形式,通过一个迭代优化进程估算了运动的准确性能。 对于强有力的跟踪性能,TAToo 采用了一种概率性格配方,并在目标水平上实施几何限制。 结果: 我们用模拟数据来验证TAToo,只要有地面真相运动和工具工具(Ttototom) 数据,其中光学追踪提供强有力的基线。 我们报告头骨和外科的亚模和毫米间跟踪方法的精确性能, 并分别在1xoo 的旋转错误上, 我们进一步说明TAT 网络如何在外科手术中同时使用。 我们的手动的轨道, 。 将显示一个工具的轨道, 将直接使用。