In this work we propose a methodology to accurately evaluate and compare the performance of efficient neural network building blocks for computer vision in a hardware-aware manner. Our comparison uses pareto fronts based on randomly sampled networks from a design space to capture the underlying accuracy/complexity trade-offs. We show that our approach allows to match the information obtained by previous comparison paradigms, but provides more insights in the relationship between hardware cost and accuracy. We use our methodology to analyze different building blocks and evaluate their performance on a range of embedded hardware platforms. This highlights the importance of benchmarking building blocks as a preselection step in the design process of a neural network. We show that choosing the right building block can speed up inference by up to a factor of 2x on specific hardware ML accelerators.
翻译:在这项工作中,我们提出了一个方法,以硬件意识的方式准确评估和比较计算机视觉高效神经网络构件的性能。我们的比较利用从设计空间随机抽样的网络来利用基于随机抽样的前沿线来捕捉基本的精确/复杂权衡。我们表明,我们的方法可以与以前比较范式所获得的信息相匹配,但能对硬件成本和准确度之间的关系有更深入的了解。我们用我们的方法分析不同的构件,并评价其在一系列嵌入硬件平台上的性能。这突出了基准构件作为神经网络设计过程中预选步骤的重要性。我们表明,选择正确的构件可以加快推论速度,具体硬件 ML 加速器的乘数高达2x。