The Jaccard index is an important similarity measure for item sets and Boolean data. On large datasets, an exact similarity computation is often infeasible for all item pairs both due to time and space constraints, giving rise to faster approximate methods. The algorithm of choice used to quickly compute the Jaccard index $\frac{\vert A \cap B \vert}{\vert A\cup B\vert}$ of two item sets $A$ and $B$ is usually a form of min-hashing. Most min-hashing schemes are maintainable in data streams processing only additions, but none are known to work when facing item-wise deletions. In this paper, we investigate scalable approximation algorithms for rational set similarities, a broad class of similarity measures including Jaccard. Motivated by a result of Chierichetti and Kumar [J. ACM 2015] who showed any rational set similarity $S$ admits a locality sensitive hashing (LSH) scheme if and only if the corresponding distance $1-S$ is a metric, we can show that there exists a space efficient summary maintaining a $(1\pm \varepsilon)$ multiplicative approximation to $1-S$ in dynamic data streams. This in turn also yields a $\varepsilon$ additive approximation of the similarity. The existence of these approximations hints at, but does not directly imply a LSH scheme in dynamic data streams. Our second and main contribution now lies in the design of such a LSH scheme maintainable in dynamic data streams. The scheme is space efficient, easy to implement and to the best of our knowledge the first of its kind able to process deletions.
翻译:Jaccar 索引是项目集和 Boolean 数据的一个重要相似度度量。 在大型数据集中, 精确相似度计算往往无法对所有项目配对都适用, 因为时间和空间的限制, 从而产生更快捷的近似方法 。 用于快速计算 Jaccar 指数$\ frac\ vert A\ chap B\ a\ a\ cap\ a\ cvert B\vert A\ cup\ vvert A\ cup B\ vert} $ 2 和 $B$, 通常是一种微量显示的形式。 在数据流处理中, 大多数微量显示系统只能维持数据流处理中的数据流中, 但当数据流中对应的距离为1- S美元, 但当面临项目删除时, 却并没有发现任何可操作的工作。 在本文件中, 我们的动态Slightal1 中, 将一个高效的近似近似近似近似近似近似近似的近似近似近似算法 。