Segmenting unseen objects is a crucial ability for the robot since it may encounter new environments during the operation. Recently, a popular solution is leveraging RGB-D features of large-scale synthetic data and directly applying the model to unseen real-world scenarios. However, even though depth data have fair generalization ability, the domain shift due to the Sim2Real gap is inevitable, which presents a key challenge to the unseen object instance segmentation (UOIS) model. To tackle this problem, we re-emphasize the adaptation process across Sim2Real domains in this paper. Specifically, we propose a framework to conduct the Fully Test-time RGB-D Embeddings Adaptation (FTEA) based on parameters of the BatchNorm layer. To construct the learning objective for test-time back-propagation, we propose a novel non-parametric entropy objective that can be implemented without explicit classification layers. Moreover, we design a cross-modality knowledge distillation module to encourage the information transfer during test time. The proposed method can be efficiently conducted with test-time images, without requiring annotations or revisiting the large-scale synthetic training data. Besides significant time savings, the proposed method consistently improves segmentation results on both overlap and boundary metrics, achieving state-of-the-art performances on two real-world RGB-D image datasets. We hope our work could draw attention to the test-time adaptation and reveal a promising direction for robot perception in unseen environments.
翻译:对机器人来说,隐形物体的剖析是关键的能力,因为它在操作期间可能会遇到新的环境。 最近,一个流行的解决方案正在利用大型合成数据的 RGB-D 嵌入式适应(FTEA) 功能,并将模型直接应用到不可见的现实世界情景中。然而,尽管深度数据具有公平的概括化能力,但由于Sim2Real 差距而发生的域变换是不可避免的,这是对隐性物体例分解(UOIS) 模式的一个关键挑战。为了解决这个问题,我们再次强调在本文中整个Sim2Real域的适应进程。具体地说,我们提出了一个框架,以基于 BatchNorm 层参数进行全时 RGB-D 嵌入式适应(FEA) 的测试时时时时时时制图像(FTEA) 。为了构建测试- 测试- 时间回溯性回溯性调整学习目标,我们提出了一个新的非参数变异性变换目标。 此外,我们设计了一个跨模式知识再更新模块模块,以鼓励在测试时段中进行信息传输。我们所提出的方法可以有效地使用测试时间图像,不需要说明或重新审视的周期的模拟模拟模拟的图像转换。