Deep Neural Networks have shown great promise on a variety of downstream applications; but their ability to adapt and generalize to new data and tasks remains a challenge. However, the ability to perform few or zero-shot adaptation to novel tasks is important for the scalability and deployment of machine learning models. It is therefore crucial to understand what makes for good, transfer-able features in deep networks that best allow for such adaptation. In this paper, we shed light on this by showing that features that are most transferable have high uniformity in the embedding space and propose a uniformity regularization scheme that encourages better transfer and feature reuse. We evaluate the regularization on its ability to facilitate adaptation to unseen tasks and data, for which we conduct a thorough experimental study covering four relevant, and distinct domains: few-shot Meta-Learning, Deep Metric Learning, Zero-Shot Domain Adaptation, as well as Out-of-Distribution classification. Across all experiments, we show that uniformity regularization consistently offers benefits over baseline methods and is able to achieve state-of-the-art performance in Deep Metric Learning and Meta-Learning.
翻译:深神经网络在各种下游应用中表现出了巨大的希望;但是,它们适应和普及新数据和任务的能力仍然是一项挑战;然而,对新任务进行少量或零点调整的能力对于机器学习模型的可扩缩和部署十分重要,因此,至关重要的是,要了解深网络中哪些是良好的、可转让的、最有利于这种适应的特征。在本文中,我们通过显示最可转让的特征在嵌入空间中具有高度的统一性,并提议一个统一规范化计划,鼓励更好地转让和特性再利用。我们评估其促进适应新任务和新任务和新任务的能力的正规化,为此,我们进行一项彻底的实验研究,涵盖四个相关和不同的领域:微小的元学习、深米学习、零热磁域适应,以及分配外分类。在所有的实验中,我们显示,统一规范化始终给基线方法带来好处,并能够实现深气象学习和元数据学习方面的最先进的业绩。