Semantic understanding of 3D point clouds is important for various robotics applications. Given that point-wise semantic annotation is expensive, in this paper, we address the challenge of learning models with extremely sparse labels. The core problem is how to leverage numerous unlabeled points. To this end, we propose a self-supervised 3D representation learning framework named viewpoint bottleneck. It optimizes a mutual-information based objective, which is applied on point clouds under different viewpoints. A principled analysis shows that viewpoint bottleneck leads to an elegant surrogate loss function that is suitable for large-scale point cloud data. Compared with former arts based upon contrastive learning, viewpoint bottleneck operates on the feature dimension instead of the sample dimension. This paradigm shift has several advantages: It is easy to implement and tune, does not need negative samples and performs better on our goal down-streaming task. We evaluate our method on the public benchmark ScanNet, under the pointly-supervised setting. We achieve the best quantitative results among comparable solutions. Meanwhile we provide an extensive qualitative inspection on various challenging scenes. They demonstrate that our models can produce fairly good scene parsing results for robotics applications. Our code, data and models will be made public.
翻译:对 3D 点云的语义理解对于各种机器人应用非常重要 。 鉴于点对点语义的语义描述非常昂贵, 我们在本文件中应对学习模型的挑战。 核心问题是如何利用许多未贴标签的点。 为此, 我们提议了一个自我监督的 3D 代表学习框架, 名为“ 观点瓶颈 ” 。 它优化了一个基于相互信息的目标, 在不同观点下应用在点云上。 一项原则性分析显示, 观点瓶颈导致一种优雅的代用流失功能, 适合大型点云数据。 与基于对比性学习的原艺术相比, 观点瓶雀在特征层面而不是样本层面运作。 这一模式转变有几个优点: 执行和调节很容易, 不需要负面的样本, 并且更好地完成我们的目标下流任务。 我们根据点对标准扫描网评估了我们的公共基准方法。 我们实现了可比较的解决方案中的最佳量化结果 。 同时, 我们对各种具有挑战性的场景进行广泛的定性检查。 它们表明, 我们的模型可以产生相当好的公开的场景代码 。