Learning and balancing time-varying loads in large-scale systems

Consider a system of $n$ parallel server pools where tasks arrive as a time-varying Poisson process. The system aims at balancing the load by using an inner control loop with an admission threshold to assign incoming tasks to server pools; as an outer control loop, a learning scheme adjusts this threshold over time in steps of $\Delta$ units, to keep it aligned with the time-varying overall load. If the fluctuations in the normalized load are smaller than $\Delta$, then we prove that the threshold settles for all large enough $n$ and balances the load when $\Delta = 1$. Our model captures a tradeoff between optimality and stability, since for higher $\Delta$ the degree of balance decreases, but the threshold remains constant under larger load fluctuations. The analysis of this model is mathematically challenging, particularly since the learning scheme relies on subtle variations in the occupancy state of the system which vanish on the fluid scale; the methodology developed in this paper overcomes this hurdle by leveraging the tractability of the specific system dynamics. Strong approximations are used to prove certain dynamical properties which are then used to characterize the behavior of the system, without relying on a traditional fluid-limit analysis.