Cost-Optimal Microservices Deployment with Cluster Autoscaling and Spot Pricing

Microservices architecture has been established as an ideal software architecture for cloud-based software development and deployment, offering many benefits such as agility and efficiency. Microservices are often associated with containers and container orchestration systems for deployment, as containerization provides convenient tools and techniques for resource management, including the automation of orchestration processes. Among the factors that make the cloud suitable for commercial software deployment, transient pricing options like AWS Spot Pricing are particularly attractive as they allow consumers to significantly reduce cloud costs. However, the dynamic nature of resource demand and the abrupt termination of spot VMs make transient pricing challenging. Nonetheless, containerization and container orchestration systems open new avenues to optimize the cost of microservices deployments by leveraging spot pricing on the public cloud while achieving application and business goals. We propose SpotKube, an open-source, Kubernetes-based, application-aware, genetic algorithm-based solution for cost optimization, which autoscales clusters for microservices-based applications hosted on public clouds with spot pricing options. SpotKube analyzes application characteristics and recommends the optimal configuration for resource allocation to the cluster. It consists of an elastic cluster autoscaler powered by an optimization algorithm that ensures cost-effective microservices deployment while meeting application performance requirements and handling abrupt termination of nodes, thereby minimizing the impact on system availability. We implement and evaluate SpotKube with representative microservices-based applications in a real public cloud setup, demonstrating the effectiveness of our approach against alternative optimization strategies.