Today, an increasing number of Adaptive Deep Neural Networks (AdNNs) are being used on resource-constrained embedded devices. We observe that, similar to traditional software, redundant computation exists in AdNNs, resulting in considerable performance degradation. The performance degradation is dependent on the input and is referred to as input-dependent performance bottlenecks (IDPBs). To ensure an AdNN satisfies the performance requirements of resource-constrained applications, it is essential to conduct performance testing to detect IDPBs in the AdNN. Existing neural network testing methods are primarily concerned with correctness testing, which does not involve performance testing. To fill this gap, we propose DeepPerform, a scalable approach to generate test samples to detect the IDPBs in AdNNs. We first demonstrate how the problem of generating performance test samples detecting IDPBs can be formulated as an optimization problem. Following that, we demonstrate how DeepPerform efficiently handles the optimization problem by learning and estimating the distribution of AdNNs' computational consumption. We evaluate DeepPerform on three widely used datasets against five popular AdNN models. The results show that DeepPerform generates test samples that cause more severe performance degradation (FLOPs: increase up to 552\%). Furthermore, DeepPerform is substantially more efficient than the baseline methods in generating test inputs(runtime overhead: only 6-10 milliseconds).
翻译:今天,越来越多的适应性深神经网络(AdNN)被用于资源限制的嵌入装置。我们观察到,与传统软件一样,在AdNNS中存在冗余计算,导致性能显著退化。性能退化取决于投入,并被称为依赖投入的性能瓶颈(IDPBs)。为确保AdNNN满足资源限制应用的性能要求,必须进行性能测试,以在AdNNT中检测国内流离失所者。现有的神经网络测试方法主要涉及正确性测试,这不涉及性能测试。为填补这一空白,我们建议采用深Perforfor,这是生成测试样本以检测AdNNS中的国内流离失所者的可扩缩的方法。我们首先展示如何将生成性能测试样本来检测国内流离失所者的性能瓶颈作为优化问题。随后,我们演示深度PNNWs计算消耗量的分布如何有效处理优化问题。我们评估了三种广泛使用的数据集的深Porforform,而这与五个流行的AdNNN模型无关。我们建议采用深Pforformat,这是一种可扩展性测试性能测试性能测试5-x 10-rodepractimp 测试方法。结果产生更大规模的性能降解。结果。DILFLAd-xmormormormormormorm