Work-in-Progress: Real-Time Neural Network Inference on a Custom RISC-V Multicore Vector Processor

Neural networks are increasingly used in real-time systems, such as automated driving applications. This requires high-performance hardware with predictable timing behavior. State-of-the-art real-time hardware is limited in memory and compute resources. On the other hand, modern accelerator systems lack the necessary predictability properties, mainly due to interference in the memory subsystem. We present a new hardware architecture with an accompanying compiler-based deployment toolchain to close this gap between performance and predictability. The hardware architecture consists of a multicore vector processor with predictable cores, each with local scratchpad memories. A central management core facilitates access to shared external memory through a static schedule calculated at compile-time. The presented compiler exploits the fixed data flow of neural networks and WCET estimates of subtasks running on individual cores to compute this schedule. Through this approach, the WCET estimate of the overall system can be obtained from the subtask WCET estimates, data transfer times, and access times of the shared memory in conjunction with the schedule calculated by the compiler.