Hybrids of Convolutional Neural Network (CNN) and Vision Transformer (ViT) have outperformed pure CNN or ViT architecture. However, since these architectures require large parameters and incur large computational costs, they are unsuitable for tinyML deployment. This paper introduces a new hybrid CNN-ViT search space for Neural Architecture Search (NAS) to find efficient hybrid architectures for image classification. The search space covers hybrid CNN and ViT blocks to learn local and global information, as well as the novel Pooling block of searchable pooling layers for efficient feature map reduction. Experimental results on the CIFAR10 dataset show that our proposed search space can produce hybrid CNN-ViT architectures with superior accuracy and inference speed to ResNet-based tinyML models under tight model size constraints.
翻译：卷积神经网络（CNN）与视觉Transformer（ViT）的混合架构在性能上已超越纯CNN或纯ViT架构。然而，由于这些架构需要大量参数并产生高计算成本，它们并不适用于微型机器学习（tinyML）部署。本文针对图像分类任务，提出了一种用于神经架构搜索（NAS）的新型混合CNN-ViT搜索空间，以寻找高效的混合架构。该搜索空间涵盖混合CNN与ViT模块，用于学习局部与全局信息，同时引入了包含可搜索池化层的新型池化模块，以实现高效的特征图降维。在CIFAR10数据集上的实验结果表明，在严格的模型大小限制下，我们提出的搜索空间能够生成优于基于ResNet的微型机器学习模型的混合CNN-ViT架构，在准确率与推理速度方面均表现出色。
相关内容
微信扫码咨询专知VIP会员