Distributed computation is always a tricky topic to deal with, especially in context of various requirements in various scenarios. A popular solution is to use Apache Spark with a setup of multiple systems forming a cluster. However, the prerequisite setup for a Spark cluster often induces an additional overhead, often limiting usage in constrained scenarios, especially in scenarios requiring context propagation. In this paper, we explore a relatively lightweight computational graph execution framework requiring little setup and fast speeds, coupled with context awareness.
翻译:分布式计算始终是一个棘手的话题,特别是在不同情况下的各种需求情况下。一个流行的解决方案是使用 *Apache Spark*,并设置多个系统形成一个集群。然而,为 *Spark* 集群进行先决条件设置时往往会带来额外的开销,通常会限制在受约束的情况下的使用,特别是在需要上下文传播的情况下。在本文中,我们探讨了一种相对轻量级的计算图执行框架,只需要很少的设置即可提高速度,同时具备上下文感知能力。