We propose two novel transferability metrics F-OTCE (Fast Optimal Transport based Conditional Entropy) and JC-OTCE (Joint Correspondence OTCE) to evaluate how much the source model (task) can benefit the learning of the target task and to learn more transferable representations for cross-domain cross-task transfer learning. Unlike the existing metric that requires evaluating the empirical transferability on auxiliary tasks, our metrics are auxiliary-free such that they can be computed much more efficiently. Specifically, F-OTCE estimates transferability by first solving an Optimal Transport (OT) problem between source and target distributions, and then uses the optimal coupling to compute the Negative Conditional Entropy between source and target labels. It can also serve as a loss function to maximize the transferability of the source model before finetuning on the target task. Meanwhile, JC-OTCE improves the transferability robustness of F-OTCE by including label distances in the OT problem, though it may incur additional computation cost. Extensive experiments demonstrate that F-OTCE and JC-OTCE outperform state-of-the-art auxiliary-free metrics by 18.85% and 28.88%, respectively in correlation coefficient with the ground-truth transfer accuracy. By eliminating the training cost of auxiliary tasks, the two metrics reduces the total computation time of the previous method from 43 minutes to 9.32s and 10.78s, respectively, for a pair of tasks. When used as a loss function, F-OTCE shows consistent improvements on the transfer accuracy of the source model in few-shot classification experiments, with up to 4.41% accuracy gain.
翻译:10. 具体而言,F-OTCE通过首先解决源与目标分布之间的最佳运输(OT)问题来估计可转让性,然后利用最佳组合来计算源模型(Task)对目标任务学习的好处,并学习更多可转让的跨域跨任务转移学习。与要求评估辅助任务的经验转移性的现有指标不同,我们的指标是无辅助性的,因此可以更高效地计算。具体地说,F-OTCE通过首先解决源与目标分布之间的最佳运输(OT)问题来估计可转让性,然后利用最佳组合来计算源与目标标签之间的负目标任务(Task),并学习更多的可转让性。它也可以起到损失功能,在调整目标任务之前,最大限度地实现源模型的可转让性。同时,JC-OTCE通过在OTCE中添加标签距离问题,尽管它可能带来额外的计算成本。