H-EYE: Holistic Resource Modeling and Management for Diversely Scaled Edge-Cloud Systems

Computing systems have been evolving to be more pervasive, heterogeneous, and dynamic. An increasing number of emerging domains now rely on diverse edge to cloud continuum where the execution of applications often spans various tiers of systems with significantly heterogeneous computational capabilities. Resources in each tier are often handled in isolation due to scalability and privacy concerns. However, better overall resource utilization could be achieved if different tiers of systems had the means to communicate their computational capabilities. In this paper, we propose H-EYE, a universal approach to holistically capture diverse computational characteristics of edge-cloud systems with arbitrary topologies and to manage the assignment of tasks to the computational resources with the whole continuum in the scope. Our proposed work introduces two significant innovations: (1) We present a multi-layer, graph-based hardware (HW) representation and a modular performance modeling interface that could capture interactions and inference between different computing and communication resources in the system at desired level of detail. (2) We introduce a novel orchestrator mechanism that leverages the graph-based HW representation to hierarchically locate target devices that a given set of tasks could be mapped to. Orchestrator provides isolation for various device groups and allows hierarchical abstraction to scalably find mappings that satisfy system deadlines. The orchestrator internally relies on a novel traverser that takes shared resource slowdown into account. We demonstrate the utility and flexibility of H-EYE on edge-server systems that are deployed on the field in two different disciplines, improving up to 47% latency over baselines with less than 2% scheduling overhead