Unsupervised feature learning often finds low-dimensional embeddings that capture the structure of complex data. For tasks for which prior expert topological knowledge is available, incorporating this into the learned representation may lead to higher quality embeddings. For example, this may help one to embed the data into a given number of clusters, or to accommodate for noise that prevents one from deriving the distribution of the data over the model directly, which can then be learned more effectively. However, a general tool for integrating different prior topological knowledge into embeddings is lacking. Although differentiable topology layers have been recently developed that can (re)shape embeddings into prespecified topological models, they have two important limitations for representation learning, which we address in this paper. First, the currently suggested topological losses fail to represent simple models such as clusters and flares in a natural manner. Second, these losses neglect all original structural (such as neighborhood) information in the data that is useful for learning. We overcome these limitations by introducing a new set of topological losses, and proposing their usage as a way for topologically regularizing data embeddings to naturally represent a prespecified model. We include thorough experiments on synthetic and real data that highlight the usefulness and versatility of this approach, with applications ranging from modeling high-dimensional single-cell data, to graph embedding.
翻译:未经监督的特征学习往往发现有低维的嵌入,可以捕捉复杂的数据结构。对于先前具备专家地貌学知识的任务,将这种知识纳入到所学的表层模型中可能会导致质量更高的嵌入。例如,这可能有助于将数据嵌入一个特定组群,或适应噪音,从而防止数据直接通过模型传播,然后可以更有效地学习这些数据。然而,缺乏将不同先前的地貌学知识纳入嵌入结构的一般工具。虽然最近开发了不同的表层,可以(重新)将数据嵌入预先确定的表层模型,但它们对于代表学习有两大限制,我们在本文件中讨论。首先,目前建议的表层损失不能代表简单的模型,例如自然方式的集群和信号。第二,这些损失忽略了数据中所有原始的结构(如邻里区)信息,而这些信息对学习有用。我们通过引入一套新的表层损失来克服这些局限性,并提议使用它们作为将数据在表层上定期嵌入数据嵌入到自然的图层模型中的一种方法,这是我们在本文件中讨论的两个重要限制。我们包括了从一个完整的模型到一个完整的模型的模型的模型。我们从一个全面的实验。