Deep classifiers have achieved great success in visual recognition. However, real-world data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this paper, we show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. Theoretically, we derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound. In addition, we introduce Balanced Meta-Softmax, applying a complementary Meta Sampler to estimate the optimal class sample rate and further improve long-tailed learning. In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.
翻译:深分类者在视觉识别方面取得了巨大成功。 然而, 真实世界数据是自然界的长尾目, 导致培训和测试分布不匹配。 在本文中, 我们显示软形函数虽然在大多数分类任务中使用, 却在长尾结构下给出了偏差梯度估计。 本文展示了平衡软形( Softmax 的优雅公正延伸), 以适应培训和测试之间的标签分布变化。 从理论上讲, 我们从多级软体回归中得出常规化, 并显示我们的损失最小化了约束 。 此外, 我们引入了平衡元体- 软体, 使用一个互补的元体样本来估计最佳类样本率, 并进一步改进长尾学习 。 在我们的实验中, 我们展示了平衡的元体- 软体( Softmax) 超越了最先进的长成型分类方法, 在视觉识别和实例分割任务上都显示我们的损失最小化了。