Cloud Matrix Machine for Julia and Implicit Parallelization for Matrix Languages

Matrix computations are widely used in increasing sizes and complexity in the fields of scientific computing and engineering. But with current matrix language implementations it is a challenging task to fully utilize Cloud compute capacities. We present a new framework called cloud matrix machine, which extends the Julia high-performance compute language to automatically parallelize matrix computations for the cloud. With this framework, users are shielded from the complexity of explicitly parallel computations. Instead, users employ a novel matrix data type with lazy evaluation semantics to facilitate implicit parallelization of matrix operations. A combination of offline profiling, dynamic simulation, and scheduling are utilized to select optimal tile sizes, schedule, and execute matrix operations. All computations occur in the Cloud, with minimal user intervention. We conducted an extensive experimental evaluation on a set of eight benchmarks using up to eight nodes (288 vCPUs) in the AWS public cloud. Our framework achieved speedups of up to a factor of 3.49x, within 20.5% of the theoretically possible maximum speedup.