Standard few-shot benchmarks are often built upon simplifying assumptions on the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks, so that the query-set classes of a given task are unknown, but just belong to a much larger set of possible classes. Our setting could be viewed as an instance of the challenging yet practical problem of extremely imbalanced K-way classification, K being much larger than the values typically used in standard benchmarks, and with potentially irrelevant supervision from the support set. Expectedly, our setting incurs drops in the performances of state-of-the-art methods. Motivated by these observations, we introduce a PrimAl Dual Minimum Description LEngth (PADDLE) formulation, which balances data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set. Our constrained MDL-like objective promotes competition among a large set of possible classes, preserving only effective classes that befit better the data of a few-shot task. It is hyperparameter free, and could be applied on top of any base-class training. Furthermore, we derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments over the standard few-shot datasets and the more realistic and challenging i-Nat dataset show highly competitive performances of our method, more so when the numbers of possible classes in the tasks increase. Our code is publicly available at https://github.com/SegoleneMartin/PADDLE.
翻译:标准点数基准往往建立在简化查询组的假设基础上,这些假设在实践上可能并不总是维持。特别是,对于测试时通常使用的每个任务来说,在未贴标签查询组中有效存在的类别都是先验的,与标签式支持组中代表的一组类别完全一致。我们放松这些假设并扩大当前基准,以便某个特定任务的查询组类别不为人知,而只是属于一系列大得多的可能类别。我们的设置可以被视为一个具有挑战性但实际的问题的例子,即极不平衡的K-way分类,K比标准基准中通常使用的复杂程度要大得多,而且可能与支持组中的数据监管无关。预期,我们的设置在最先进的支持组方法中,我们采用的是PrimAl-Dal 最低描述 LEngth (PADDLE) 公式,它平衡了某项微小任务的数据准确性和模型复杂性,在支持组的监督制约下,我们受制约的MDL-类似目标组目标组中推动了大量的竞争力竞争,而数据级数可能与此无关。在最先进的标准级级中,只有高标准级的排序中,我们可以保留一个更高的标准级级,在最高级任务中可以展示。