EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation

Achieving low remote memory access latency remains the biggest challenge in realizing memory disaggregation over Ethernet inside the datacenter. We present EDM that tries to overcome this challenge using two key ideas. First, while the existing network protocols for remote memory access over Ethernet, such as TCP/IP and RDMA, are implemented on top of Ethernet's MAC layer, EDM takes a rather radical approach of implementing the entire network protocol stack for remote memory access within the Physical layer (PHY) of the Ethernet. This overcomes fundamental latency and bandwidth overheads imposed by the MAC layer, especially for small memory messages. Second, EDM implements a centralized, fast, in-network traffic scheduler for memory traffic within the PHY of the Ethernet switch. Inspired by the classic Parallel Iterative Matching (PIM) algorithm, the scheduler dynamically reserves bandwidth between compute and memory nodes by creating virtual circuits in the switch's PHY, thus eliminating the queuing delay and layer 2 packet processing delay at the switch for memory traffic, with high bandwidth utilization. Our FPGA testbed shows that EDM's network fabric incurs a latency of only $\sim$300 ns for remote memory access in an unloaded network, which is an order of magnitude lower than state-of-the-art Ethernet-based solutions such as RoCEv2 and comparable to the emerging PCIe-based solutions such as CXL. Larger-scale network simulations show that even at high network loads, EDM's latency is within 1.3$\times$ its unloaded latency.