Handling skew is one of the major challenges in query processing. In distributed computational environments such as MapReduce, uneven distribution of the data to the servers is not desired. One of the dominant measures that we want to optimize in distributed environments is communication cost. In a MapReduce job this is the amount of data that is transferred from the mappers to the reducers. In this paper we will introduce a novel technique for handling skew when we want to compute a multiway join in one MapReduce round with minimum communication cost. This technique is actually an adaptation of the Shares algorithm [Afrati et. al, TKDE 2011].
翻译:处理 skew 是查询处理的主要挑战之一 。 在分布式计算环境中, 如 MapReduce, 数据向服务器的分布不均不可取 。 我们希望在分布式环境中优化的主要措施之一是通信成本 。 在映射中, 任务就是从绘图器向递减器传输数据的数量 。 在本文中, 当我们想要计算一个多条路连接到一个配有最低通信成本的 MapRuece 回合时, 我们将会引入一种处理扭曲的新技术 。 这个技术实际上是对 Shares 算法的调整 [Afrati et. al., TKDE 2011] 。