Wave: A Split OS Architecture for Application Engines

The end of Moore's Law and the tightening performance requirements in today's clouds make re-architecting the software stack a necessity. To address this, cloud providers and vendors offload the virtualization control plane and data plane, along with the host OS data plane, to IPUs (SmartNICs), recovering scarce host resources that are then used by applications. However, the host OS control plane--encompassing kernel thread scheduling, memory management, the network stack, file systems, and more--is left on the host CPU and degrades workload performance. This paper presents Wave, a split OS architecture that moves OS subsystem policies to the IPU while keeping OS mechanisms on the host CPU. Wave not only frees host CPU resources, but it reduces host workload interference and leverages network insights on the IPU to improve policy decisions. Wave makes OS control plane offloading practical despite high host-IPU communication latency, lack of a coherent interconnect, and operation across two system images. We present Wave's design and implementation, and implement several OS subsystems in Wave, including kernel thread scheduling, the control plane for a network stack, and memory management. We then evaluate the Wave subsystems on Stubby (scheduling and network), our GCE VM service (scheduling), and RocksDB (memory management and scheduling). We demonstrate that Wave subsystems are competitive with and often superior to on-host subsystems, saving 8 host CPUs for Stubby, 16 host CPUs for database memory management, and improving VM performance by up to 11.2%.