Data mixing augmentation have proved to be effective for improving the generalization ability of deep neural networks. While early methods mix samples by hand-crafted policies (e.g., linear interpolation), recent methods utilize saliency information to match the mixed samples and labels via complex offline optimization. However, there arises a trade-off between precise mixing policies and optimization complexity. To address this challenge, we propose a novel automatic mixup (AutoMix) framework, where the mixup policy is parameterized and serves the ultimate classification goal directly. Specifically, AutoMix reformulates the mixup classification into two sub-tasks (i.e., mixed sample generation and mixup classification) with corresponding sub-networks and solves them in a bi-level optimization framework. For the generation, a learnable lightweight mixup generator, Mix Block, is designed to generate mixed samples by modeling patch-wise relationships under the direct supervision of the corresponding mixed labels. To prevent the degradation and instability of bi-level optimization, we further introduce a momentum pipeline to train AutoMix in an end-to-end manner. Extensive experiments on nine image benchmarks prove the superiority of AutoMix compared with state-of-the-arts in various classification scenarios and downstream tasks.
翻译:混合数据增强已证明对提高深神经网络的普及能力十分有效。早期方法通过手工制作的政策(如线性内插)混合样本,但最近的方法利用突出信息,通过复杂的离线优化来匹配混合样本和标签;然而,精确混合政策和优化复杂性之间产生了一种权衡。为了应对这一挑战,我们提议了一个新型自动混合(Automix)框架,其中混合政策是参数化的,直接服务于最终分类目标。具体地说,AutoMix将混合分类分为两个子任务(即混合样本生成和混合分类),同时采用相应的子网络,在双级优化框架内解决这些问题。对于生成者,一个可学习的轻量性混合生成器,Mix Block,目的是通过在相应的混合标签的直接监督下建模补丁关系来生成混合样本。为了防止双级优化的退化和不稳定,我们进一步引入了动力管道,以最终到端的方式培训自动混合的分任务(即混合样本生成和混合分类)和在双级优化框架内解决这些问题。关于九种图像基准的大规模实验和下游任务,证明了自治的高度。