Relative Lempel-Ziv (RLZ) parsing is a dictionary compression method in which a string $S$ is compressed relative to a second string $R$ (called the reference) by parsing $S$ into a sequence of substrings that occur in $R$. RLZ is particularly effective at compressing sets of strings that have a high degree of similarity to the reference string, such as a set of genomes of individuals from the same species. With the now cheap cost of DNA sequencing, such data sets have become extremely abundant and are rapidly growing. In this paper, instead of using a single reference string for the entire collection, we investigate the use of different reference strings for subsets of the collection, with the aim of improving compression. In particular, we form a rooted tree (or hierarchy) on the strings and then compressed each string using RLZ with parent as reference, storing only the root of the tree in plain text. To decompress, we traverse the tree in BFS order starting at the root, decompressing children with respect to their parent. We show that this approach leads to a twofold improvement in compression on bacterial genome data sets, with negligible effect on decompression time compared to the standard single reference approach. We show that an effective hierarchy for a given set of strings can be constructed by computing the optimal arborescence of a completed weighted digraph of the strings, with weights as the number of phrases in the RLZ parsing of the source and destination vertices. We further show that instead of computing the complete graph, a sparse graph derived using locality sensitive hashing can significantly reduce the cost of computing a good hierarchy, without adversely effecting compression performance.
翻译:相对的 Lempel- Ziv (RLZ) 剖析是一种字典压缩方法, 将字符串 $S$压缩为第二字符串 $R$ (称为引用) 。 在本文中, 将字符串压缩为以$为单位的子字符串序列。 特别是, RLZ 特别有效地压缩与参考字符串高度相似的字符串组, 如一组来自同一物种的个人基因组。 由于现在DNA测序成本低廉, 此类数据集已经变得非常丰富且正在迅速增长。 在本文中, 而不是对整个收藏使用单一的引用字符串 $R$( 称为引用) 。 我们调查了将 $S$( 引用) 压缩成以美元为单位的子字符串序列序列序列序列序列。 我们通过不使用双倍的直径直径直的直径直值排序, 也可以通过直径直的平面的直径直的平方根线将每根直线串压缩。