Automated augmentation is an emerging and effective technique to search for data augmentation policies to improve generalizability of deep neural network training. Most existing work focuses on constructing a unified policy applicable to all data samples in a given dataset, without considering sample or class variations. In this paper, we propose a novel two-stage data augmentation algorithm, named Label-Aware AutoAugment (LA3), which takes advantage of the label information, and learns augmentation policies separately for samples of different labels. LA3 consists of two learning stages, where in the first stage, individual augmentation methods are evaluated and ranked for each label via Bayesian Optimization aided by a neural predictor, which allows us to identify effective augmentation techniques for each label under a low search cost. And in the second stage, a composite augmentation policy is constructed out of a selection of effective as well as complementary augmentations, which produces significant performance boost and can be easily deployed in typical model training. Extensive experiments demonstrate that LA3 achieves excellent performance matching or surpassing existing methods on CIFAR-10 and CIFAR-100, and achieves a new state-of-the-art ImageNet accuracy of 79.97% on ResNet-50 among auto-augmentation methods, while maintaining a low computational cost.
翻译:自动增强是一种新兴而有效的技术,用于搜索数据增强策略,以提高深度神经网络训练的泛化能力。大多数现有的工作都着眼于构建适用于给定数据集中所有数据样本的统一增强策略,而不考虑数据样本或类别变化。本文提出了一种新颖的两阶段数据增强算法,称为标签感知自动增强(LA3),它利用标签信息,为不同标签的样本分别学习增强策略。LA3包括两个学习阶段,在第一阶段中,通过一种神经预测器辅助的贝叶斯优化,评估并排名每个标签的单独增强方法,以便我们在低搜索成本下确定每个标签的有效增强技术。在第二阶段中,除了有效的增强技术,还构建了一种复合增强策略,从而产生显著的性能提升,并可以在典型的模型训练中轻松部署。广泛的实验表明,LA3在CIFAR-10和CIFAR-100上实现了出色的性能匹配或超越现有方法,并在自动增强方法中在ResNet-50上实现了新的ImageNet准确性纪录:79.97%,同时保持低的计算成本。