Semantic segmentation in 3D indoor scenes has achieved remarkable performance under the supervision of large-scale annotated data. However, previous works rely on the assumption that the training and testing data are of the same distribution, which may suffer from performance degradation when evaluated on the out-of-distribution scenes. To alleviate the annotation cost and the performance degradation, this paper introduces the synthetic-to-real domain generalization setting to this task. Specifically, the domain gap between synthetic and real-world point cloud data mainly lies in the different layouts and point patterns. To address these problems, we first propose a clustering instance mix (CINMix) augmentation technique to diversify the layouts of the source data. In addition, we augment the point patterns of the source data and introduce non-parametric multi-prototypes to ameliorate the intra-class variance enlarged by the augmented point patterns. The multi-prototypes can model the intra-class variance and rectify the global classifier in both training and inference stages. Experiments on the synthetic-to-real benchmark demonstrate that both CINMix and multi-prototypes can narrow the distribution gap and thus improve the generalization ability on real-world datasets.
翻译:在大规模附加说明的数据的监督下,3D室内场景的语义分解取得了显著的成绩,然而,先前的工作依据的假设是,培训和测试数据分布相同,在分配范围外的场景上进行评估时,可能因性能退化而受到影响。为减轻批注成本和性能退化,本文件介绍了这项任务的合成到现实域概括性设置。具体地说,合成和现实世界点云数据之间的域间差距主要在于不同的布局和点模式。为了解决这些问题,我们首先提议采用集成实例组合(CINMix)增强技术,使源数据布局多样化。此外,我们扩大源数据模式,采用非参数性多模型,以缓解因增加点模式而扩大的阶级内部差异。多模型可以模拟阶级内部差异,并在培训和推断阶段纠正全球归分级者。关于合成到现实基准的实验表明,CINMix和多模型类型都能够缩小分布差距,从而改进现实世界数据的总化能力。