Efficient Bypass in Mesh and Torus NoCs

Minimizing latency and power are key goals in the design of NoC routers. Different proposals combine lookahead routing and router bypass to skip the arbitration and buffering, reducing router delay. However, the conditions to use the bypass require completely empty buffers in the intermediate routers. This restricts the amount of flits that use the bypass pipeline especially at medium and high loads, increasing latency and power. This paper presents and reviews NEBB, Non-Empty Buffer Bypass, a mechanism that allows to bypass flits even if the buffers to bypass are not empty. The mechanism applies to wormhole and virtual-cut-through, each of them with different advantages. NEBB-Hybrid is proposed to employ the best flow control in each situation. The mechanism is extended to torus topologies, using FBFC and shared buffers. The proposals have been evaluated using Booksim, showing up to 75% reduction of the buffered flits for single-flit packets, which translates into latency and dynamic power reductions of up to 30% and 23% respectively. For bimodal traffic, these improvements are 20 and 21% respectively. Additionally, the bypass utilization is largely independent of the number of VCs when using shared buffers and very competitive with few private ones, allowing to simplify the allocation mechanisms.