DNN workloads can be scheduled onto DNN accelerators in many different ways: from layer-by-layer scheduling to cross-layer depth-first scheduling (a.k.a. layer fusion, or cascaded execution). This results in a very broad scheduling space, with each schedule leading to varying hardware (HW) costs in terms of energy and latency. To rapidly explore this vast space for a wide variety of hardware architectures, analytical cost models are crucial to estimate scheduling effects on the HW level. However, state-of-the-art cost models are lacking support for exploring the complete depth-first scheduling space, for instance focusing only on activations while ignoring weights, or modeling only DRAM accesses while overlooking on-chip data movements. These limitations prevent researchers from systematically and accurately understanding the depth-first scheduling space. After formalizing this design space, this work proposes a unified modeling framework, DeFiNES, for layer-by-layer and depth-first scheduling to fill in the gaps. DeFiNES enables analytically estimating the hardware cost for possible schedules in terms of both energy and latency, while considering data access at every memory level. This is done for each schedule and HW architecture under study by optimally choosing the active part of the memory hierarchy per unique combination of operand, layer, and feature map tile. The hardware costs are estimated, taking into account both data computation and data copy phases. The analytical cost model is validated against measured data from a taped-out depth-first DNN accelerator, DepFiN, showing good modeling accuracy at the end-to-end neural network level. A comparison with generalized state-of-the-art demonstrates up to 10X better solutions found with DeFiNES.
翻译:DNN 工作量可以通过多种不同方式被排在 DNN 加速器上:从逐层排期到跨层深度排期(a.k.a.a. 层融合,或级推执行),这导致一个非常广泛的排程空间,每个排程导致不同的硬件(HW)在能量和延缓度方面成本不同。为了迅速探索这一广阔的空间以各种硬件结构进行广泛的验证,分析成本模型对于估计对 HW 水平的排程影响至关重要。然而,最先进的通用成本模型缺乏对探索整个深度第一排排排排位空间的支持,例如只侧重于启动而忽略权重,或仅建模 DRA 访问,而忽视芯图数据动态数据移动。这些限制使得研究人员无法系统、准确地理解第一层排排位的排程空间。在将这一设计空间正规化后,这项工作提出了一个统一的模型框架,即DeFINERS,用于逐层和深度排期排程,以填补空白。DFIONS 能够分析估算从能量和平流水平上可能的排程的排程空间,同时考虑调试算数据在每部分,在每层的运行级的轨道结构中,在每层数据流中将显示每个存储数据流数据流数据流、每层的计算。