In this article we present our relocatable distributed collections library. Building on top of the AGPAS for Java library, we provide a number of useful intra-node parallel patterns as well as the features necessary to support the distributed nature of the computation through clearly identified methods. In particular, the transfer of distributed collections' entries between processes is supported via an integrated relocation system. This enables dynamic load-balancing capabilities, making it possible for programs to adapt to uneven or evolving cluster performance. The system we developed makes it possible to dynamically control the distribution and the data-flow of distributed programs through high-level abstractions. Programmers using our library can therefore write complex distributed programs combining computation and communication phases through a consistent API. We evaluate the performance of our library against two programs taken from well-known Java benchmark suites, demonstrating superior programmability, and obtaining better performance on one benchmark and reasonable overhead on the second. Finally, we demonstrate the ease and benefits of load-balancing and on a more complex application which uses the various features of our library extensively.
翻译:在文章中,我们展示了我们可移动分布的收藏图书馆。在为爪哇图书馆提供的AGPAS之上,我们提供了一些有用的节内平行模式,以及支持通过明确确定的方法进行计算分布性质的必要特征。特别是,通过一个综合迁移系统,支持在两个程序之间转移分布的收藏条目。这能够产生动态的负载平衡能力,使各程序能够适应不均衡或不断演变的集群性能。我们开发的系统使得能够通过高层次的抽象性来动态控制分布式程序的分配和数据流。因此,我们图书馆的程序员可以通过一个一致的API来编写复杂的分布式程序,将计算和通信阶段结合起来。我们对照从著名的爪哇基准套件中取来的两个方案,评估图书馆的业绩,展示了优异的可编程性,并在第二个基准和合理的间接费用上取得更好的性能。最后,我们展示了负荷平衡和更为复杂的应用程序的便利和好处,这些应用程序广泛利用了我们图书馆的各种特征。